Synchronizing the Profiles database with your organization's user data
To keep your organization's user data up-to-date, regularly synchronize the IBM® Connections Profiles database with your data source, such as an LDAP directory or an employee database.
The Profiles database is the core repository within Connections for information about the Connections users in your organization. Typically, the primary source of this user data is your organization's LDAP directory, but you can also synchronize from non-LDAP sources, either exclusively or in combination with an LDAP directory. For more information about synchronizing non-LDAP sources, see Using a custom source repository connector.
The synchronization process is controlled by properties in the profiles_tdi.properties file.
It is recommended that you use the sync_all_dns command to transfer changes in your organization's user data repository to the Profiles database. If you want to keep the Profiles database in a close synchronized state with your LDAP directory, run this task nightly or at another frequency that suits you.
Note: An alternative approach is to synchronize by using a change log that is maintained by the LDAP. However, there are significant challenges to getting this approach to work, and it is not recommended. For more information about using this approach, see Synchronizing IBM Tivoli Directory Server and Microsoft Active Directory LDAP changes.
During synchronization, the values of attributes that are mapped from the LDAP directory to the Profiles database in the file map_dbrepos_from_source.properties are evaluated to determine which users need updating. In the Profiles database, existing users are updated, deleted, or deactivated, and new users are created. If you configure extension attributes, that data is also compared and synchronized. The comparison includes all user data within the search scope, including extension data.
The sync_all_dns command processes one user at a time, and as the number of users and extension attributes increases, daily synchronization can take too long. There are two performance-related options for sync_all_dns that you can use: multi-processing, and time stamp tracking. The multi-processing option divides up the work into independent processes that proceed simultaneously so that multiple users are processed concurrently. The time stamp tracking option tracks the LDAP time of last update, which all LDAPs support, and eliminates the need for most comparing. For details about these options, see Improving the performance of the sync_all_dns command. There are certain circumstances where the time stamp cannot be observed, for example, when you are switching LDAP directories.
The sync_all_dns command creates temporary files that are used during the synchronization process. Use the sync_updates_working_directory property to specify the location of the temporary files. Use the sync_updates_clean_temp_files property to specify whether to delete or retain the temporary files after synchronization. Retaining the files is useful when you are troubleshooting a problem.
In addition to the temporary files, the following files in the TDI solution directory record the changes that were made during synchronization:
When the sync_updates_show_summary_only property is set to true, no changes are made.
Like the other TDI tasks, the sync_all_dns command writes log information to the log file ibmdi.log in the TDI\logs directory. You can check the log to see whether the command finishes successfully, and look for error information if necessary.
For more information about how the sync_all_dns command works, see Understanding how the sync_all_dns process works.
To synchronize LDAP directory changes with Profiles:
Use the properties in the following table to control the synchronization process.
Option Description sync_updates_hash_field sync_updates_hash_field is a key property. This property specifies the field that is used to match a user record in the Profiles database with the corresponding user information in the source. The supported fields are uid, guid, and email. The default is uid.
It is critical that you choose a field that does not ordinarily change over time, so that the match will remain intact. If the value in the field in the source does change, the match is broken, and the existing database information for this person could be deleted.
If the value of the hash field in the source does change, you must set this property to a different field that has not changed, for at least one run of sync_all_dns. For example, if the value for uid changes in the source, you must set the property to either guid or email. After one run of sync_all_dns, you can change the property back to uid.
Note: If the value is guid and you change LDAP providers, you must change the value to uid or email temporarily because guid is LDAP-specific.
perform_deletion_or_inactivate_for_sync The default value is true. Set this property to false when you don't want to delete or mark as inactive those users who are no longer in the LDAP directory.
The sync_all_dns command checks the value of the property and acts by using the following logic:
- If the value is true, look at the sync_delete_or_inactivate property to determine which action to take. The action is either delete or inactivate.
- If the value is false, perform neither the delete action nor the inactivate action.
sync_delete_or_inactivate Controls what happens to a user record when it is not found in the LDAP directory. The value must be either delete or inactivate, and is case-sensitive. Inactivate is basically a soft delete. By default, the property is set to inactivate. The inactive state is propagated to all the other Connections applications independent of whether the user is deleted or inactivated.
For information about hard deleting users who have been inactive for some time, see Using supplied scripts to delete inactive users based on inactivity length.
For more information about how to use this property, see Populating a large user set.
This property is not configurable when you are using the population wizard.
source_ldap_iterate_with_filter_functions_file Used only when source_ldap_iterate_with_filter is set to true.
Use when the size of the data to be retrieved from LDAP exceeds the search limit of the LDAP or exceeds the memory capacity of the TDI LDAP connector. For example, if your search parameters would return 250 K records but your LDAP only allows 10K to be returned at a time, you can use this property.
For more information about how to use this property, see Populating a large user set.
This property is not configurable when using the population wizard.
sync_updates_double_check The default value is false. When set to true, the assembly line that is defined by the sync_check_if_remove property runs.
Note: This property applies to delete/inactivate only.
sync_check_if_remove Used only when sync_updates_double_check is set to true.
Specifies the name of an assembly line in profiles_tdi.xml that verifies the delete operation or the inactivate operation.
By default, the name of the assembly line is set to sync_all_dns_check_if_remove. The sync_all_dns_check_if_remove assembly line looks up the distinguished name of the about-to-be-deleted user in the LDAP directory. If the user is found, sync_all_dns_check_if_remove returns a status that causes the main assembly to bypass the delete or inactivate action.
For more information about this property, see Customizing the logic used for the delete operation.
sync_updates_clean_temp_files The default value is true. When set to false, temporary files are not deleted until the next time that sync_all_dns is run. sync_updates_hash_partitions Number of partitions to divide the temporary files into. The default of 10 is sufficient in most cases. If problems develop, you can increase the value. The typical problem is running out of memory during the update phase because all the data related to all users in a partition is held in memory during the update, delete, and add phases. For more information about partitions, see Understanding how the sync_all_dns process works. sync_updates_show_summary_only The default value is false. When set to true, the employee.* files in the TDI solution directory contain the records that are changed, but no changes are made. sync_updates_working_directory The directory where the working files are stored. The path can be relative to the TDI solution directory or an absolute path. The default value is sync_updates, which is a relative path. sync_updates_size_model The default value is single. This property is used for enhancing the performance of the sync_all_dns command. Possible values are single, multi4, multi6, or multi8. For more information about this property, see Improving the performance of the sync_all_dns command sync_updates_use_ldap_timestamp The default value is false. This property is used for enhancing the performance of the sync_all_dns command. For more information about this property, see Improving the performance of the sync_all_dns command.
If you are storing data from multiple LDAP branches or multiple LDAP directories in the same Profiles database, you must synchronize each LDAP branch or LDAP directory separately. To accomplish this task, you can set the following properties in the profiles_tdi.properties file.
Note: These properties can only be used with the sync_all_dns command. They cannot be used with the process_tds_changes and process_ad_changes commands.
|sync_source_url_enforce|The default value is false. When set to true, synchronizes only those users where the stored source URL matches the current source URL. The current source URL is the concatenation of the source_ldap_url, source_ldap_search_base, and source_ldap_search_filter properties. That is, it limits the scope of the set of data in the database, and skips the records that do not match the current source URL.| |sync_source_url_override|The default value is false. This property is effective only when sync_source_url_enforce is true. When sync_source_url_enforce is true and sync_source_url_override is false, records where the current source URL and the stored source URL do not match are skipped.
When sync_source_url_override is true, the records that would have been skipped are checked for a match of the hash field in the current LDAP branch or LDAP directory. If there is a match and at least one field needs to be updated, the record is updated, and the source URL is set to the current value. If no fields are updated, which would be very unusual in the cases where you would use this override, no change is made. For example, the major use case is switching LDAP directories, and in this case the guid is sure to change.
This property should be clearly understood before setting it to true. See the last example later in this document to see it in action.| |sync_store_source_url|The default value is true. Stores the source LDAP URL in the prof_source_url field in the database. The source LDAP URL is needed to determine the source of the data to correctly synchronize it when there is more than one LDAP branch or LDAP directory. Even if there is only one LDAP, it is best to leave this property set to true.|
Run the sync_all_dns command. The command name is either sync_all_dns.sh or sync_all_dns.bat, depending on your operating system.
Note: When the sync_all_dns command runs, a lock file is created in the TDI solution directory. The lock file prevents others from starting a sync_all_dns process in the same TDI solution directory. The name of the lock file is sync_all_dns.lck. The lock file is deleted after the sync_all_dns command completes. If the command does not complete, the lock file is not deleted. You can delete it yourself, or you can run the clearLock.sh or the clearLock.bat script, located in the TDI solution directory.
Example employee tables
The sample EMPLOYEE table illustrates results from a scenario for the fictional Zeta Bank company, in which you have pulled users A, B, and C from the Littleton LDAP branch and users X, Y, and Z from the Westford LDAP branch.
Following the best practice, you have used two TDI solution directories with slightly different profiles_tdi.properties files. In the following discussion, one of the solution directories is called ABC, and the other is called XYZ.
Each of the profiles_tdi.properties files include the following entries:
source_ldap_url=ldap://ldap.zetabank.com:389 source_ldap_search_filter=((objectClass=inetOrgPerson)(uid=*)) sync_store_source_url=true sync_source_url_enforce=true sync_source_url_override=false
The ABC/profiles_tdi.properties file, for the Littleton branch, includes the following entry:
The XYZ/profiles_tdi.properties file, for the Westford branch, includes the following entry:
After running collect_dns and populate_from_dn_file in each of the two TDI solution directories, the EMPLOYEE table contains the following data:
Table 1. Example employee table after running collect_dns and populate_from_dn_file in each TDI solution directory.
Notice that the only difference between the values for PROF_SOURCE_URL is the location parameter.
If you run sync_all_dns in the ABC TDI solution directory, you get updates for people A, B, and C, but not for people X, Y, and Z because sync_source_url_enforce is set to true and the PROF_SOURCE_URL for those people does not match the concatenation of the source_ldap_url, source_ldap_search_base, and source_ldap_search_filter properties. Setting sync_source_url_enforce to false causes the people X, Y, and Z to be deleted from the database because they don’t exist in the Littleton branch.
If the people A, B, and C move from Littleton to Waltham, the ABC/profiles_tdi.properties would then have the following entry for source_ldap_search_base:
An update is required because the location value in the PROF_SOURCE_URL column is different than the location value in ABC/profiles_tdi.properties. To correctly update the value in PROF_SOURCE_URL, in ABC/profiles_tdi.properties set sync_source_url_override to true and then run sync_all_dns in the ABC TDI solution directory.
As a safety precaution, the PROF_SOURCE_URL is not updated if it is the only attribute that changes. Also, you should set sync_source_url_override to false after running sync_all_dns.
After the command completes, the EMPLOYEE table contains the following data:
Table 2. Example employee table after running collect_dns in the ABC TDI solution directory.
- Synchronizing IBM Tivoli Security Directory Server and Microsoft Active Directory LDAP changes
To keep your profiles synchronized with your LDAP directory, use the generic sync_all_dns command. However, if your LDAP directory is Tivoli Security Directory Server or Microsoft® Active Directory, you can use the process_tds_changes or process_ad_changes commands. You must configure your LDAP server to save all updates to a change log, which places a considerable burden on the LDAP, and you must run the change log server.
- Synchronizing user data between Profiles and the LDAP directory
When you allow users to make changes to certain fields in their profiles, the changes need to be copied from the Profiles database to the LDAP directory, if the field value originates in the LDAP directory.
- Understanding how the sync_all_dns process works
Use the sync_all_dns process to keep your Connections Profiles database synchronized with changes to the LDAP directory.
- Improving the performance of the sync_all_dns command
Significantly improve the performance of the Connections Profiles sync_all_dns command by enabling two properties in the profiles_tdi.properties file.