Import job definition

You can import user profiles from the ReachFive Console. This makes using existing user profiles that you may already have easier as well as enabling you to batch update users via a JSON or CSV file.

This page shows you all you need to know regarding formatting for importing profiles as well as the validation process, import and merge logic, and import limitations in the following sections:

For instructions on how to create and run an import job from the ReachFive Console, check out the Import profiles page.
Sequential imports

Imports are sequential. This means that an import file can create and update the same profile during an import.

For example: if user A with the name Marie is created early in the import and later on in the same import file, user A has the name updated to Maria. When the import is done, user A keeps Maria as her name.

Import format

Here, we look at how you need to format import files.

JSON

The schema for the imported user is based off of the user profile object. In this format, all fields are optional.

{
  "external_id": "1",
  "email": "foo@gmail.com"
} \n (1)
{
  "email": "bar@gmail.com",
  "name": "Joe",
  "gender": "M",
  "custom_fields": { (2)
      "has_loyalty_card": true
  },
  "consents": { (3)
    "exampleConsent": {
      "date": "2021-11-23T11:42:40.858Z",
      "consent_version": {
        "language": "fr",
        "version_id": 1
      },
      "granted": true,
      "consent_type": "opt-in",
      "reporter": "managed"
    }
  }
}
1 Each profile must separated by a line feed character (\n).
2 Import custom_fields as an object containing a series of fields.
3 Import consents as an object or as a flattened field with the format consents.<consent>.<parameter> shown below. 
...
  "consents.cgu.date": "2021-09-03T19:08:01Z",
  "consents.cgu.granted": true,
  "consents.cgu.consent_version.version_id": 2,
  "consents.cgu.consent_version.language": "fr",
  "consents.cgu.consent_type": "opt-in",
  "consents.cgu.reporter": "managed"
...

CSV

The CSV format uses a free format where columns are flattened JSON paths.

For example a path { "foo" : { "bar" : "test" } }` becomes foo.bar in the CSV header.

If a column has no value, then the corresponding field will not be used for the current line. 
email,name,gender,custom_fields.has_loyalty_card,consents.newsletter.consent_type,consents.newsletter.granted,consents.newsletter.date,consents.newsletter.reporter
bar@gmail.com,Joe,M,true,opt-in,true,2018-05-25T15:41:09.671Z,managed
File validation

Once parsed, every imported profile is validated to ensure that only consistent data is imported.

Profile import files must comply with the following requirements:

  • At least one unique field should be provided (several can be provided):

    • email (verified or unverified),

    • phone_number (if the SMS feature is available, verified or unverified),

    • a valid provider

    • a valid external_id

  • At most a single matching profile should be returned for merging. The matching is made on:

    • uid (corresponding to the profile id),

    • email (verified or unverified),

    • phone_number (if the SMS feature is available, verified or unverified),

    • provider

  • Only one other profile can match the imported profile. If more than one user is matched, we can’t automatically define how to merge them, so an error will be returned.

  • Any provided consent should be defined for an existing key with a date anterior to the current date.

  • Any provided custom field should be defined for an existing key.

  • Any provided password hash should specify a valid hash method.

Import and merge logic

This section details various import and merge logic details for import jobs.

Subsections:

Delete existing fields

If you want to delete an existing field, you need to pass a null value as part of the import file.

  • The updated_at value of the import file must be more recent than the existing profile.

  • Currently, you cannot delete addresses.

  • JSON

  • CSV

{
  "updated_at": "2021-06-04T14:16:34.658Z",
  "email": "bar@gmail.com",
  "name": "Joe",
  "family_name": null, (1)
  "gender": "M",
  "identities": [ { "provider": "facebook", "user_id": "123" } ]
}
1 Pass null to delete the existing family_name field. 
external_id,updated_at,email,name,family_name,gender,identities.0.provider,identities.0.user_id
1,2021-06-04T14:16:34.658Z,foo@gmail.com,,,,,,bar@gmail.com,Joe,__null__,M,facebook,123 (1)
1 Pass __null__ to delete the existing family_name field.

Create a user via import file

You can create a user via an import file. When you create a user this way, the created_at and updated_at fields are automatically set to the profile creation date.

If the created_at and/or updated_at fields are already present in the file for the particular user being created, the dates in the file are used.

Import A = You create a user and the file has no created_at or updated_at fields. The profile creation date is used for the fields.

Import B = You create a user and the file does contain the created_at or updated_at fields. The dates currently in the file are used for the fields.

Security

You can provide password hashes during an import under the password_hash field. We are able to develop new algorithms to ensure full compatibility with your database. When a plain text password is used, the password is hashed during the import using the Bcrypt algorithm. When hashed passwords are used, the password is hashed using Bcrypt only after the user’s first login.

This is only available for users who have never logged in.

If a user has, in fact, already logged in at least once, the password_hash field in an updated user import job will remain the same and not be updated. In other words, the old password will remain and is not updated.

The following algoritms are allowed:

Algorithm Notes

bcrypt

md5

Can be used with optional salt and iterations parameters where salt is used as a prefix.

sha1

Can be used with optional salt parameter used as a postfix.

sha256

Can be used with optional salt and iterations parameters where salt is used as a prefix.

sha512

Can be used with optional salt parameter used as a postfix.

sha512Prefixed

Can be used with optional prefix and salt parameters where prefix used as a prefix and salt is used as a postfix.

drupalSha512

Cannot be used with optional parameters.

sha256PostSalt

Can be used with optional salt and iterations parameters where salt is used as a postfix.

magentoSha256

Cannot be used with optional parameters.

magento

Support for hashes from magento in the form hash:salt:v1:v2 with v1 and v2 being the hashes applied in succession (when the hashes have been migrated.)

No other parameters are taken into account as the hash contains everything needed.
Use this instead of magentoSha256 if you require a more generic approach.

plaintext

 
{
  "prefix": "...",
  "value": "...",
  "algorithm": "sha512Prefixed",
  "salt": "..."
}
email,password_hash.value,password_hash.algorithm,password_hash.iterations,password_hash.salt
emmanuel.scholt@example.com,$S$5T/ViD4qlx6qHTQAci8eEbCuP/MVsoO8OSfU.xJsykqT5H6BXowl,sha256PostSalt,10,azdAAZeQw

Encrypt import file

When importing files with ReachFive, it is inherently secure and encrypted. However, you may wish to provide additional encryption for the import process.

To encrypt your desired import file, run the following command from the root of the file location:

openssl aes-256-cbc -salt -pbkdf2 -iter 10000 -in file_to_import -out file_to_import.enc
Command section Notes

-salt

Salts the password.

-pbkdf2

Declares use of PBKDF2 function.

-iter

Specifies the number of iterations of PBKDF2.

Example: -iter 10000

-in file_to_import

The name of the original file to import.

Example: -in file_to_import

-out file_to_import.enc

The name of the output file (encrypted).

Example: -out file_to_import.enc

Social Login

You can import multiple social login profiles using the identities field. The provider name must be a valid provider name, and the user_id field should be the ID of the user from the provider’s site.

The user’s profile will be populated with provider information after the first login of the user using the corresponding social provider.

Merge

If a corresponding profile is found during a merge, the profile will not be created but merged with the existing profile. The matching is based on the following fields:

  • id (corresponding to the profile id),

  • email (verified or unverified),

  • phone_number (if the SMS feature is available, verified or unverified),

  • provider

A profile should never match more than one other profile, or an error will be returned, since there is no way to know the merge priority in a 3+ way merge.

The merge is a safe merge, meaning that (following the priority) no field is overwritten. Fields can either be completed when they are lists or free objects (consents, custom_fields), created if empty, or completely ignored if already defined.

Merge priority

The priority of the merge is defined by the attribute updated_at. If the attribute is not present on both profiles, then the existing profile has priority over the imported profile.

Consents are the exception. If a consent is present in both merged profiles, the unified profile takes the consent with the most recent value based on the consent’s date attribute.

Import limitations

There is no way to define to which provider an email or phone_number belongs to. Keep in mind that after the first login using a social provider, the profile’s information will be updated using the provider’s own information, completing the missing fields of the profile.

Force data update from file

The Force data update from file option is available on the ReachFive Console when importing profiles. This option does not use the typical merge logic for user profiles but instead forces the data you import to update all existing user profile fields.

The Force data update from file option does not apply to the consent date. This will always reflect the date at which the consent decision occurred.
There may be unwanted behaviour if you use this option so be sure you understand your requirements before choosing this approach. See the import and merge logic section for more details on standard merge logic.
How does it work?

If there are fields with values in the current ReachFive profile that are not in the import, these fields retain their current value. However, all fields with values replace the existing fields in ReachFive.

force data update from file

Import profiles in Testing mode

import profiles test mode

When you import profiles in Testing mode, ReachFive tests:

  • access to the server

  • access to the import file

  • whether the the import file content is valid

Testing mode does not handle database modifications.

Import Lite profiles only

import lite profiles

When you import Lite profiles only, you are not importing the fully managed profiles, but rather those lite profiles that registered with your site. You have to import Lite profiles separately from managed profiles.