CategoriesIdentity

Simple MIM Sync Run Profiles Scheduler using PowerShell and Task Scheduler – GG EZ

Today I’m going to share one weird trick to schedule MIM Run Profiles using PowerShell. Consultants hate me. Let’s get started.

Script

We are going to create two folders. The first folder is where we are going to store the PowerShell script and configuration. The second folder is where we are going to store our logfiles. I will use C:\MIMRunScheduler and C:\MIMRunScheduler\Log.

param([string]$configfile=$(Read-Host -prompt "Configfile"))

### Load Configuration Data ###
[xml]$maconfig=(Get-Content $configfile)
"" > $maconfig.runcycle.logfile
$sourceFolder="C:\MIMRunScheduler\"
### Functions ###
$line = "---------------------------------------------------------------------------------"
function Write-Output-Banner([string]$msg) {
       Write-Output $line,(" "+$msg),$line
    Write-Output $line,(" "+$msg),$line | Add-Content $sourceFolder"Log\$(Get-Date -f yyyy-MM-dd).log"
}

function Write-Output-Text([string]$msg) {
       Write-Output $msg
    Write-Output $msg | Add-Content $sourceFolder"Log\$(Get-Date -f yyyy-MM-dd).log"
}


### Get Management Agent Data ###
$allMA = @(get-wmiobject -class "MIIS_ManagementAgent" -namespace "root\MicrosoftIdentityIntegrationServer" -computername $maconfig.runcycle.computername)
$numOfExecDone = 0


### Main Script ###
do {
       Write-Output-Banner("Execution #:"+(++$numOfExecDone)+" - Date: "+(date))
       foreach($MANextRun in $maconfig.runcycle.ma) {
             $found = $false;
             
             foreach($MA in $allMA) {   
               if(!$found) {
                    if($MA.Name.Equals($MANextRun.name)) {
                           $found=$true;
                           Write-Output-Banner("MA: "+$MA.Name+" [Type: "+$MA.Type+"]")
                           Write-Output-Text(" - Starting Pre-Script: "+$MANextRun.preScript)
                           invoke-expression $MANextRun.preScript | Out-Null
                           Write-Output-Text(" - Done`n")
                           
                           foreach($profileName in $MANextRun.profilesToRun) {
                                  Write-Output-Text(" - Starting Profile: "+$profileName)
                                  $datetimeBefore = Get-Date;
                                  $result = $MA.Execute($profileName);
                                  $datetimeAfter = Get-Date;
                                  $duration = $datetimeAfter - $datetimeBefore;
                                  Write-Output-Text(" - Done with status: "+$result.ReturnValue+" - Duration: "+$duration.Hours+":"+$duration.Minutes+":"+$duration.Seconds+"`n")
                           }
                           
                           Write-Output-Text(" - Starting Post-Script: "+$MANextRun.postScript)
                           invoke-expression $MANextRun.postScript | Out-Null
                           Write-Output-Text(" - Done`n")
                           Start-Sleep -s $MANextRun.waitSeconds
                    }
               }
             }
             if(!$found) { Write-Output-Text("Not found MA name :"+$MANextRun.name); }
       }

       $continue = ($maconfig.runcycle.cycleWaitSeconds -EQ 0) -OR ($numOfExecDone -lt $maconfig.runcycle.numOfExecs) -OR ($maconfig.runcycle.numOfExecs -EQ -1)
       if($continue) { 
             Write-Output-Banner("Sleeping "+$maconfig.runcycle.cycleWaitSeconds+" seconds")
             Start-Sleep -s $maconfig.runcycle.cycleWaitSeconds
       }
} while($continue)

Save the PowerShell script as C:\MIMRunScheduler\MIMRunCycle.ps1.

The PowerShell script accepts an XML configuration file as a parameter and causes the MIM Synchronization Engine runs to be performed. How the runs are performed is described in the configuration file.

Within the PowerShell script, there is a $sourceFolder variable. This is responsible for the location where the log files will reside.

Script Syntax

.\MIMRunCycle.ps1 [-configFile] <location>

-configFile <location>

The <location> refers to the location of the configuration file. The following example shows how this can be done:

-configfile C:\MIMRunScheduler\FULLConfig.XML

NameValue
Required?True
Accept wildcard characters?False
Accept pipeline input?False
PositionNamed

Configuration

We are going to create two additional scripts called C:\MIMRunScheduler\pre.ps1 and C:\MIMRunScheduler\post.ps1. If you want to perform an action right before or after a run, you can use these two scripts. We are going to leave them empty.

<?xml version="1.0" encoding="UTF-8"?>
<runcycle numOfExecs="0" cycleWaitSeconds="15" computerName=".">
   <MA>
      <name>ADMA</name>
      <preScript>C:\MIMRunScheduler\pre.ps1</preScript>
      <profilesToRun>FIFS</profilesToRun>
      <postScript>C:\MIMRunScheduler\post.ps1</postScript>
      <waitSeconds>5</waitSeconds>
   </MA>
   <MA>
      <name>MIMMA</name>
      <preScript>C:\MIMRunScheduler\pre.ps1</preScript>
      <profilesToRun>FIFS</profilesToRun>
      <postScript>C:\MIMRunScheduler\post.ps1</postScript>
      <waitSeconds>5</waitSeconds>
   </MA>
</runcycle>

Save the configuration file as C:\MIMRunScheduler\FULLConfig.xml.

Configuration Syntax

Each configuration file starts with the XML declaration.

<?xml version="1.0" encoding="UTF-8"?>

After that, run cycles (runcycle in XML) can be defined. A runcycle contains three parameters.

<runcycle numOfExecs="0" cycleWaitSeconds="15" computerName=".">
NameDatatypeDescription
numOfExecsInteger0 = perform the run once
-1 = perform the runs in perpetuity
>0 = define how often the runs should be performed
cycleWaitSecondsIntegerTime between performing run cycles in seconds.
computerNameStringSpecify the computer name in FQDN or in NetBIOS on which MIM is installed. Use the value “.” if the PowerShell script is running on the same server on which MIM is installed.

In a runcycle, multiple MA elements (<MA>) can be defined. In each MA element contains the following elements.

   <MA>
      <name>ADMA</name>
      <preScript>C:\MIMRunScheduler\pre.ps1</preScript>
      <profilesToRun>FIFS</profilesToRun>
      <postScript>C:\MIMRunScheduler\post.ps1</postScript>
      <waitSeconds>5</waitSeconds>
   </MA>
NameDatatypeDescription
nameStringSpecify the name of the Management Agent.
preScriptStringSpecify the location of the PowerShell script to be performed before the run.
profilesToRunStringSpecify the run profile of the selected Management Agent from the name element.
postScriptStringSpecify the location of the PowerShell script to be performed after the run.
waitSecondsIntegerSpecify how many seconds to wait between running the Management Agents.

Task Scheduler

Create a new task in Windows Task Scheduler.

In the General tab, ensure that the task is run by a service account with sufficient rights in MIMSync. Furthermore, pay attention to the Local Security PolicyLocal PoliciesUser Rights Management > Log on as a batch job permission.

Go to the Actions tab.

Press New… to create a new action.

Action:Start a program
Program/script:Powershell.exe “C:\MIMRunScheduler\MIMRunCycle.ps1”
Add arguments (optional):-configfile “C:\MIMRunScheduler\FULLConfig.XML”

Click on OK to save.

Side note: In some environments, the PowerShell execution policy is set to restricted so make sure that’s dealt with first with the system administrator and security officer.

Action:Start a program
Program/script:Powershell.exe -ExecutionPolicy Bypass “C:\MIMRunScheduler\MIMRunCycle.ps1”
Add arguments (optional):-configfile “C:\MIMRunScheduler\FULLConfig.XML”
Disclaimer: Use -ExecutionPolicy Bypass at your own discretion. CISO’s hate this one trick.

Save and run the task.

Conclusion

That’s all folks! I really prefer this method of scheduling MIMSync runs because the script is very lightweight and easy to understand. Hope that you can make use of this. Have a good one.

CategoriesIdentity

FIM Service Management Agent (MIMMA) is not responsive and ends with a stopped-server error – [sadness noises]

The stopped-server error is difficult to troubleshoot because of its indistinct error reporting and multiple causes. It is basically saying that something has happened but Microsoft Identity Manager Synchronization Service (MIMSync) doesn’t know what exactly. It is indicating that the error is beyond the scope of MIMSync. We should focus on what the MIMMA is trying to do in order to troubleshoot this correctly.

Let’s take a look at the architecture of MIM in general. The architecture diagram is based on FIM but this still apply to MIM. As for naming convention we’ll continue to use MIMSync and MIMService.

FIM Architecture Diagram by Brad Turner and David Lundell

We can see that the MIMSync is located in the application tier between the MIMService and the database servers. Most of the time the MIMService is also installed on the MIMPortal server but as shown in the diagram, they are two separate entities. This means that if I disable every Microsoft Internet Information Services (IIS) instances that is hosting MIMPortal, MIMService will continue to run as this is a stand-alone web service (Microsoft.ResourceManagement.Service.exe). As for the scope of this problem, we have established that the stopped-server error doesn’t involve MIMPortal nor IIS.

The MIMMA behaves like a SQLMA with additional ECMA/XMA properties. It connects to the database server and to the MIMService’s base address. The export run in particular connects to the MIMService. Should the MIMService endpoint be unavailable, the export will generate the following error.

An error occurred in executing a Web service object request. 
Type: System.ServiceModel.EndpointNotFoundException 

Message: There was no endpoint listening at http://mim.identandy.local:5725/ResourceManagementService/MEX that could accept the message. This is often caused by an incorrect address or SOAP action. See InnerException, if present, for more details.

Stack Trace:    at System.Net.HttpWebRequest.GetRequestStream(TransportContext& context)
   at System.Net.HttpWebRequest.GetRequestStream()
   at System.ServiceModel.Channels.HttpOutput.WebRequestHttpOutput.GetOutputStream()

Inner Exception: Unable to connect to the remote server

Troubleshooting

There are multiple causes that could make the stopped-server error occur. I have categorized them as the following four suspects:

  • The MIMService is unreachable. We need to establish what is preventing the connection between MIMSync and MIMService.
    • Check the Windows Services Manager (services.msc) whether or not MIMService is running.
    • If the MIMService isn’t running, check if the service account has sufficient rights.
    • Check the firewall whether or not port 5725 is blocked.
    • Check the connectivity to the MIMService endpoint using a browser. It should display a default WCF services page.
  • The SQL Server is unreachable. We need to confirm that the management agent can connect to the SQL database and server.
    • Check the SQL Server Configuration Manager whether or not SQL Server and the concerning instance are running.
    • Check the firewall if the necessary ports are open.
    • Check the SQL Server Client Network Utility (cliconfg.exe) whether or not the correct network library and server name is being used. Normally I would use TCP/IP with “Dynamically determine port” enabled.
    • Check the SQL Server Management Studio (SSMS) whether or not the service account has login permissions.
  • The SQL Server is unresponsive. We can assume that MIMSync can successfully (or partially) connect to the database server.
    • Check the Event Viewer for SQL timeout exceptions such as “Net SqlClient Data Provider: System.Data.SqlClient.SqlException: Timeout expired. The timeout period elapsed prior to completion of the operation or the server is not responding“.
      • This indicates that there are performance problems that needs to be addressed.
    • Check the Event Viewer for SQL permission exceptions such as “System.Data.SqlClient.SqlException: The EXECUTE permission was denied on the object 'StoredProcedureName', database 'FIMService', schema 'sync'.“.
      • This should be easily fixed by adding the service account to the db_owner role of the FIMService database.
    • Check the Event Viewer for SQL tempdb exceptions such as “Could not allocate space for object ‘dbo.SORT temporary run storage: ***************‘ in database ‘tempdb’ because the ‘PRIMARY’ filegroup is full. Create disk space by deleting unneeded files, dropping objects in the filegroup, adding additional files to the filegroup, or setting autogrowth on for existing files in the filegroup.”.
      • While I haven’t fully established the source of this problem, I have managed to work around this by stopping the MIMSync and MIMService instances and clearing the TempDB’s by either a SQL Server restart or via SQL scripts.
  • The MIMMA, MIMService or SQL Server is taking forever to complete requests. This one is by far the most difficult to troubleshoot.
    • Check if the implementation is following the best practices defined by Microsoft.
    • Check the SQL Server Management Studio (SSMS) to find deadlocks in the FIMService and FIMSynchronization databases.
    • Check the Management Policy Rules (MPR) in MIM Portal to find contradicting rules or erroneous workflows.
    • Check if the server infrastructure is adequately equipped. Ensure that the SQL Server has enough IOPS, memory and CPU to limit bottlenecking.

If you are still experiencing problems, consider getting a MIM health check done by a Microsoft Professional to discover underlying issues that might be the cause of this problem. Contact me for more information.

CategoriesIdentity

Unable to disconnect a connector in MIM – Errors… Errors everywhere.

I was greeted with an ambiguous error that occured in Microsoft Identity Manager (MIM) Synchronization Service Manager. (MIMSync) The error message in question:

[SYNCHRONIZATION SERVICE MANAGER]
Could not find any resources appropriate for the specified culture or the neutral culture.
Make sure Microsoft.DirectoryServices.MetadirectoryServices.UI.PropertySheetBase.MMSErrorMessages.resources" was correctly embedded or linked into assembly "PropertySheetBase" at compile time, or that all the satellite assemblies required are loadable and fully signed. [OK]

Let’s start from the beginning. I was trying to disconnect a connector from the connector space in MIMSync and I was greeted with a warning:

[SYNCHRONIZATION SERVICE MANAGER]
You are about to disconnect an unconfirmed provisioned connector or an object that is a pending import delete. This operation will immediately delete the object from connector space. Do you want to continue? [YES/NO]

Sure, I wanted to make it a disconnector but having the object disappear completely would be a viable alternative as well. I clicked on [YES] and then I was shown the “Could not find any resources…” error. What’s going on? The error indicates as if something is wrong with MIMSync itself but it could also be something entirely else.

Upon looking further I’ve found that the object/connector that I was trying to disconnect is having a synchronization error. The synchronization error was “connector-filter-rule-violation” under the synchronization step: “Connector filter (export)”. Then I generated a Full Synchronization preview and here’s what I’ve found in the preview; under the MIM Service management agent, there’s a match with a defined connection filter rule. This is what is causing all the problems!

I opened the MIM Service management agent (MIMMA) and checked the Connector Filter configuration. I found the matched filter that was causing the problem and I deleted it. I went back to the Metaverse Search and opened the connector I wanted to disconnect. Clicked on Disconnect, another “You are about to disconnect an unconfirmed provisioned…” error, clicked on YES and it’s successfully disconnected! I did not have to perform any Import/Syncs/Exports after deleting the connection filter. After that’s done, I restored the connection filter back on the MIMMA.

Job done. ✅

Update (2021-06-27) I’ve noticed that the “Could not find any resources appropriate…” error is directly related to a synchronization error on a management agent. This is because I’ve seen the error happen again but this time, it wasn’t a “connection-filter-rule-violation” error. So if you could fix the synchronization error on one of the connectors of the object you’re trying to disconnect, that will fix the problem.