SharePoint Online and Office 365 groups: What is the difference between soft delete and hard delete?

Yes, SharePoint Online and Office 365 groups (aka Unified Group) are concurrent. Example, if you have create a Microsoft Teams Group/Team or create an Office 365 groups or create a Team sites, Microsoft Office 365 will create a site or create a group for it too. However, now Microsoft Office 365 has a new category of create site, called Communication site. (FYI, I am still learning on it).
Anyway, I am here to provide a notice or knowledge to technical and end users, when comes to creating and editing/modification sites, everyone are very good at it. However, when it comes to deletion, no one really cares !
Now here comes the interesting part, try asking yourself “What actually happens, when i deleted a group?” and your possible answer is “Oh, it just went into my recycle bin.”. I couldn’t agree more to that answer but that is only partial information that you know about the deletion function.

○ Definition of soft deletion – “Recycle bin with retention period of 30 days”
○ Definition of hard deletion – “After retention period of 30 days and 93 days, Microsoft Online Service Garbage Collector (You will notice this from your Audit log in Security and Compliance), will automatically permanently force delete anything inside the recycle bin”
*Note:
○ This may takes up to 24 hours or 48 hours for permanent deletion completion

Anyway, administrator and users don’t have to be afraid of 30 days retention, where now Microsoft has extended the retention to 93 days, where all things in recycle bin will be move to a secondary recycle bin. However, there is always a limited quota for secondary recycle bin, if exceeds than it will automatically purges the oldest items. (Please refer to the references)
References;

Outlook: Why am I seeing this pop-up frequently? How do I check the causes?

If you have an environment that is newly deploy or newly upgraded and after few months, than you have only encounter your Outlook frequently shows the pop-up for credentials. Thus, you have no idea, how come there would be such issue happen, even though you have deploy it with best practice.

To check what is causing this situation is to run wireshark;

  1. Run wireshark on a user’s computer, either connected to LAN or Internet.
    • Close all necessary applications
    • Open and run the wireshark
    • Open Outlook only
    • If no pop-up shown, than open other microsoft applications, such as excel or skype for business
    • If than pop up shown, than stop and save your wireshark logs
    • Analyse the wireshark logs
    • You will probably see there is multiple re-transmission of the firewall connection, which successful and than fail instantly
    • This could be your firewall issue that causes the pop up
  2. You can also check from Event viewer from the user’s computer, based on the similar steps for wireshark

For such situation happen, the only assumption you will ask yourself is;

  1. Has there be changes with firewall?
  2. Is the firewall having issue?
  3. Is my exchange or exchange hybrid having issue?
  4. Is my ADFS having problem? (You can check from portal access is it accessible, if yes than ADFS is not the problem)

SharePoint Online: Why you should not Share your main site to external? What are the best practice?

As an organization, anything that is internal stays internal and if anything needs to be share to external, are only provided view permission to only specific site or documents, especially organization’s SharePoint Online or On premises main sites.

Providing sharing option for external is dangerous as this will causes sudden surprise of deleted sites or deleted documents, and users will start to compliant asking and demanding “How come that my site get deleted?” and another disadvantage is that even “Security & Compliance’s Audit Logging & SharePoint Audit Logging”, will not provide you the details of whom has perform the operation, because allowing anonymous to access to your organization’s main sites or any other private sites will not have result shown in audit logging.

For private sites or department sites, SharePoint Online share site permission, by default is Edit. Thus, if this falls on the hands of external, he or she can have the rights to delete or modify anything within the organization’s sites and copy any Private & confidential documents and exploit your data.

*Note:

  • Office 365 group is equal to SharePoint private site.
  • Only Owner of site has permission to perform deletion
  • If audit logging is not enable on the site, activities will not shown in Security & Compliance and SharePoint Online Audit report
  • By default, audit logging is disable for private sites

Best practices;

  1. Set sharing for Main site to “Allow only internal”
  2. Try to make use of OneDrive for document sharing
  3. Enable Rights Management Service for SharePoint Online
  4. Anything internal, Stays internal
  5. Educate users on the risk of sharing to external parties
  6. Enabled Audit logging for all private sites (Only owner of the sites has permission)
    • Without this you Global admin has no visibility to that site’s behavior, even with Security & Compliance
  7. Restrict users from creating Office 365 groups (optional)

 

Office 365 & Outlook: Policy tip of “this sender fail fraud detection (spoof)”

The only reason of warning you see “sender fail fraud detection” in your outlook while sending or replying a mail to another domain, could be that the other domain doesn’t have the valid/existence of a “include” record or etc. in their SPF record. Thus, authentication fail.

To resolve this is to add the missing particular record to your SPF record;

Example:

Actual;

v=spf1 ip4:192.168.1.1 ipv4:192.168.1.2 mx ~all

 

Expected;

v=spf1 include:spf.protection.outlook.com ip4:192.168.1.1 ipv4:192.168.1.2 mx ~all

*Note: This will take up to 24 hours for the record fully propagated. There is not much differences for your “all”  with symbol of “~”, “-“, or “?”. After propagated, make use of this online DNS checking tool called “mxtoolbox” to make sure the record is correct.

If after adding/modifying your SPF record, and you still having trouble with it, is best that you check with your hosting company and raise a case to Microsoft Office 365 support  for further assistance. 

Office 365 & AD & Exchange Hybrid: How to create remote mailbox in Exchange Hybrid for existing user, in Active Directory and Office 365?

When you have existing user active directory record and you’ve accidentally had provision the mailbox at Office 365. Thus, result you unable to add the user into any distribution group and etc. because it doesn’t have record in Exchange Hybrid. Besides, user’s primary email address wasn’t correct, such as “xxxx@domain.onmicrosoft.com” instead of “xxxx@domain.com”.

Here are the steps to resolve your problems;

Implication: None (for me)

*Note: You have to be familiar with PowerShell. Best to try it on a test user account first.

  1. Go to Exchange Hybrid server
  2. Open Exchange Powershell Management
  3. Type the following commands;

    Enable-remotemailbox “userDisplayName” -RemoteRoutingAddress “xxxx@domain.mail.onmicrosoft.com”

  4. Go to Azure AD Server
  5. Open Windows Powershell

    Start-ADSyncSyncCycle -PolicyType Delta

  6. You will than review that particular user’s the mailbox in Office 365, has more email addresses shown in the email address category itself. And also the Primary email address has change to the right one.

 

*Note: This may take half an hour for the overall settings to be propagated at the user side. Because at the user side they will still view their primary smtp as the incorrect one, even though the modification has done.

Active Directory: How to export Active Directory User with all attributes?

I know that the below command will not be as effective but it does the job.

Step-by-Steps

  1. Go to Active Directory/Domain Controller
  2. Open Powershell as administrator
  3. Type the following command below;

Get-ADUser -Properties * -Filter * | Export-csv “ADUserattributes.csv”

OR

Get-ADUser -Filter * -Properties * | Export-csv “ADUserattributes.csv”

 

The above command will export the list of AD Users with attributes and values in a csv file format, and all you need to do is to copy out the attributes and paste it in a new excel file, format it from column view to row view(optional).

*Note: You could modify the command as you wish

 

Office 365 & Exchange Online: Why have to be careful with Mail Flow Rules?

Many thought that even they are from programmer background or any Technical background could achieve to implement 101% accurately correct mail flow rules, based on a programmer mindset. However, by having such thoughts will bring risks to yourself and the company itself.

*Note: If you’re not familiar of the product, then please try not to play around with it in production environment. Always, start off with research and lab testing.

Mail flow rules brings huge impact to your organization’s mail flow with slight incorrect configuration of a rule. Example, mails send to only Department B, aren’t suppose for Department A to be view but then Department A somehow able to view mails that is for Department B. However, this causes data leak within organization.

 

Is best to spend time to understand the product than rather getting yourself in trouble.

 

Reference

  1. https://technet.microsoft.com/en-us/library/jj919235(v=exchg.150).aspx

 

Office 365 Custom DLP: How to create custom Sensitive Information?

Yes this is interesting topic for me because it involve programming! I will make this topic as simple as learning alphabet, because I will be showing you the importance on how to create your very own DLP sensitive information. DLP templates are come in the form of xml file format.

*Note: You may need to spend some time in this. Practice makes perfect result.

Importance that you must include into your xml are:

  1. Rule
  2. Entity
  3. Pattern
  4. Identity Match/Id Match/Format/RegularExpression

#This is the flow chart

Rule -> Entity -> Pattern -> Identity Match/Format/Regular Expression

Ok, now you know what are the importance, next will be things you need to take note on the “importance” that can have multiple section. That is “Pattern” & “Identity Match”. You can only have 1 rule consist with 1 Entity, where that Entity can have multiple unique pattern types and each pattern can have its own unique ID Match.


Below is a sample of my code on how it looks like in xml;

*Note: You have to change the GUID of the highlighted red parts, as you can see there are 4 GUIDs, but only 2 GUID are the same. To get new GUID, you simple have to open your PowerShell and type the command “[guid]::newguid()”.

<?xml version="1.0" encoding="UTF-8"?>
<RulePackage xmlns="http://schemas.microsoft.com/office/2011/mce">
<!-- Need to change guid, rule package guid, [guid]::newguid()-->
<RulePack id="872155dc-1234-4e3e-a10d-x"> 
<Version build="0" major="1" minor="0" revision="0"/> 
<!-- Need to change guid, publisher guid --> 
<Publisher id="6907d14a-1234-4023-87cd-x"/> 
<Details defaultLangCode="en-us"> <LocalizedDetails langcode="en-us"> <PublisherName>Company Group</PublisherName> 
<Name>ID Custom Rule Pack</Name>
<Description> This rule package contains the custom ID entity. </Description> </LocalizedDetails> 
</Details>
</RulePack>

<!--This orange part, is your rule type-->
<Rules>
<!-- ID --> 
<!--This blue part, is your entity-->
<!-- need to change guid, entity guid--> 
<Entity id="b660289d-189e-1234-9e0a-x" patternsProximity="300" recommendedConfidence="70">
<!--This green part, is your pattern type-->
<Pattern confidenceLevel="80"> 
<!--This purple part, is your Identity match name-->
<IdMatch idRef="Regex_id1"/> 
</Pattern>
<Pattern confidenceLevel="80">
<IdMatch idRef="Regex_id2"/> 
</Pattern> 
</Entity>

<!--This pink part is your Regular Expression-->
<!--Format: AB-C-DE-FGH--> 
<Regex id="Regex_id1">(\d{2})[-](\d{1})[-](\d{2})[-](\d{3})</Regex> 
<!--Format: ABCDEFGMANNN -->
<Regex id="Regex_id2">(\d{7})[mM][a-zA-Z](\d{3})</Regex> 

<LocalizedStrings> 
<!-- Resource guid same as rule guid --> 
<Resource idRef="b660289d-189e-1234-9e0a-x">
<Name default="true" langcode="en-us">ID</Name>
<Description default="true" langcode="en-us"> A custom classification for detecting IDs. </Description> 
</Resource> 
</LocalizedStrings>

</Rules>

</RulePackage>

The above xml consist of 2 patterns both are set with accuracy of 80%, means if DLP scanned your mail/sharepoint/onedrive consist what is inside the pattern and has 80% match percentage will trigger the rule. Inside each pattern consist unique identity which name “Regex_id1”  and “Regex_id2”. After that, is comes to setting the format for each unique identities. As you can see above, the format i had state in the comment.

*Note: The code above doesn’t limit your needs, you could play around with what you wish to include, such as keywords, false positive, or etc.. You could learn more about twerking around the codes by reading below references. You can also use any online tester site to test out regex of your code .


References:

  1. https://support.office.com/en-us/article/Create-a-custom-sensitive-information-type-82c382a5-b6db-44fd-995d-b333b3c7fc30
  2. https://docs.microsoft.com/en-us/dotnet/standard/base-types/regular-expression-language-quick-reference
  3. https://justaucguy.wordpress.com/2014/11/21/adventures-in-custom-dlp-rules-part-one/

Office 365: Email address with random numbers attached UPN (user123@contoso.onmicrosoft.com)

Why do some of my users have random numbers attach with their UPN? Well the only answer to this question is, duplication occur because you haven’t force delete the old account from Office 365 recycle bin. And by default there is a setting in Azure AD to detect duplication is set to false, so this is another reason that even there are duplication in AD created and a sync has been trigger the duplication will also be sync to the Office 365.

To prevent any duplication in future to be sync to the Office 365, is to set the duplication checking in Azure AD to true. So that any duplication is scan will be rejected to be sync to the Office 365. To resolve the duplication currently having is to delete the account and resync it to Office 365.

*Note: These random numbers in your smtp has no implication or effects to your mailbox/account, but it is only not nice to see it that way. It is up to your choice to do it or not.

*Azure Module Power Shell needed

  1. Open Azure power shell
  2. Type the following command to get the sync feature

Connect-MsolService

Get-MsolDirSyncFeatures

dirsyncfeature

3. Next, enable both of the feature to true

Set-MsolDirSyncFeatures -Feature DuplicateUPNResiliency -Enabled $true

Set-MsolDirSyncFeatures – Feature DuplicateProxyAddressResiliency -Enabled $true


*Note:

Situation 1: If the mailbox is still new, you will only need to delete the account and force delete from the recycle bin, then do a resync 

Situation 2: If user have the mailbox for very long, you have to break the dirsync wait for the account’s status change from “Sync from on premise” to “on cloud“, then from Office 365 you can edit its smtp address but make sure you have your recycle bin clear from the old account. (Yes this is much troublesome than Situation 1)


Step by step for situation 1

  1. Locate the account from Office 356
  2. Unassign the mailbox license
  3. At Active Directory, move the account to a unsync organization unit
  4. At Azure AD Connect, run a manual sync command

Start-ADSyncSyncCycle -PolicyType Delta

5. Make sure the account at Office 365 has gone to the “Deleted User” Category

6. Once the account has been appear in “Deleted User” Office 365, you have to run a command to force deleted from the recycle bin.

7. Open Azure Module PowerShell

8. Type the following command, enter the UPN that you wish to remove from recycle bin

Connect-MsolService

Get-MsolUser -ReturnDeletedUser

Get-MsolUser -ReturnDeletedUser | Remove-MsolUser -UserPrincipalName xxxx@domain.com -RemoveFromRecycleBin -Force

9. After finish remove the account from recycle bin, move back the account from unsync Organization Unit to Sync Organization Unit

10. At Azure AD Connect, run the manual sync command

11. At Office 365, locate the account and assign license

12. You can notice there is no more random numbers found


Step by Step for Situation 2

*Make sure recycle bin is clear from duplication accounts

  1. Open Azure Module PowerShell
  2. Type the following command, enter the UPN that you wish to remove from recycle bin

    Connect-MsolService

    Get-MsolUser -ReturnDeletedUser

    Get-MsolUser -ReturnDeletedUser | Remove-MsolUser -UserPrincipalName xxxx@domain.com -RemoveFromRecycleBin -Force

  3. Open Azure Module Powershell
  4. Type the following command

Connect-MsolService

Set-MsolDirSyncEnabled -EnableDirSync $false

3. You probably have to wait for 24hr to 48hrs for the dirsync to complete progress

4. At Office 365 > Exchange Online > Recipients > Mailbox

5. Search for the account > Double click on it

6. Go to Email Option > select the smtp address to edit

7. Remove the random numbers from the smtp address

8. Save your changes

*Make sure you put this to lab test before trying it out in actual environment, just to get clear understanding what are you doing

 

References:

  1. https://support.office.com/en-us/article/Turn-off-directory-synchronization-for-Office-365-ee5f861e-bd48-4267-83d1-a4ead4b4a00d