Flat File Usage Tracking Instructions

ClientSuccess can support the import of aggregate usage data of your customers from "flat file".

The intended use case for a flat file import is to support Customers that are not able to integrate via the JavaScript library or push data the Usage API.

The aggregated data format requires one row per organization/user combination per day where the event metric has already been aggregated by the source system.    For example, the total # of logins for a given organization would represent a single line for the day with a sum of the total logins for that day. If user-level event data is to be tracked a similar method should be used with one row per organization per user per day.

The frequency of ingesting this file is approximately every hour, though may not be at the exact same time every hour.

Required Steps from Customer side

1. Folder has been set up and is available via S3, Google Cloud Files, or SFTP
2. Username/password has been provided (if SFTP) to support@clientsuccess.com 
3. Folder that will be used has been created and is in place
4. At least 1 production-ready sample file is in place - this file could represent real data or a few events for a ‘test’ event that can be discarded later
5. File has been validated for format (Date format including time, Org ID and Org Name are required, user fields are required in header but can be blank, value is provided (not empty) for at least 1 event.
6. Metric/Event columns must have a numeric value i.e. "0" and cannot be left blank. Do not send files with empty metric columns. This will cause the file to fail to process.
7. ClientSuccess support will then respond to validate that file ingestion process was successfully configured and initial file was successfully ingested.

"If the S3 bucket you are providing uses custom encryption keys additional configuration is required. Please reference this article (https://aws.amazon.com/premiumsupport/knowledge-center/cross-account-access-denied-error-s3/) and provide the ARN for the encryption key to your CSM or support when requesting setup."

File Format

The required file format is a CSV with the fields identified in the example below.  It is expected that proper escaping of string literals will be applied and that the first row is always the column header.

You can click to download an example template here.

*Note:  The time format typically should use the 2018-01-01T00:00:00+00:00 with the "....+00:00" left at zeros.  This is a timezone offset in the UTC time stamp format that is typically not needed.

Required Formatting:

1.) Please provide empty columns for user columns where user-level information is not in use with no values/blanks. Do not provide empty strings ("").

2.) Please ensure columns are listed in proper order/sequence starting with event.time, org.id, etc. as described below.

3.) File size is limited to 50,000 events. Anything exceeding this amount should be separated into different files.  

*Note: An event does NOT represent a single row of data in the flat file.  Each row could (and usually do) contain more than one event.  Blank cells do not count as an event.

4.) Please follow the event.time should be in ISO 8601 described below. 

Example File Format from a comma-delimited file:

event.time,org.id,org.name,user.id,user.name,user.email,login,user_login,survey_sent,survey_response,admin_action

2018-03-01T00:00:00+00:00,"100","Test Organization 1",,,,1000,,2500,250,

2018-03-01T00:00:00+00:00,"200","Test Organization 2",,,,1000,,2500,250,

2018-03-01T00:00:00+00:00,"300","Test Organization 3",,,,1000,,2500,250,

2018-03-01T00:00:00+00:00,"100","Test Organization 1","500","John Doe","john.doe@example.com",,10,2500,250,28

2018-03-01T00:00:00+00:00,"100","Test Organization 1","1000","Example User","example.user@example.com",,100,2500,250,56

Example of Template File you MUST send to your CSM for validation before configuring the S3 or SFTP folder.  Template can be downloaded from here.

File Naming:

It is recommended that files be named according to a convention that includes the date that the file contains data for.

Example: 2018-01-01-ClientSuccess-Usage.csv

Note: This example is referring to the date that the flat file would be sent.  It does not indicated that a daily flat file should be submitted.  We suggest a cadence of no more than weekly if you are going to submit a flat file for usage tracking.

Transport Options

Amazon S3

Note: S3 server is needed only if you are planning to upload a recurring flat file for regular usage tracking

Instructions:

1. Provide access to the clientsuccess-usage-bot IAM user via ARN.

The ClientSuccess ARN is: arn:aws:iam::990004405076:user/clientsuccess-aws-bot

2. Send your S3 bucket policy to support.

Below is an example of what your S3 bucket policy may look like, and what we need you to provide to us to configure the auto-ingestion process of your usage flat file from your S3 bucket:

{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Access-to-clientsuccess-aws-bot",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::990004405076:user/clientsuccess-aws-bot"
},
"Action": [
"s3:ListBucket",
"s3:GetObject"
],
"Resource": [
"arn:aws:s3:::<your bucket name here>",
"arn:aws:s3:::<your bucket name here>/*"
]
}
]
}

Google Cloud Storage

Provide access to the clientsuccess-api-bot IAM user to your bucket.

clientsuccess-api-bot@clientsuccess-prod.iam.gserviceaccount.com

Secured FTP folder

Provide access with a specific user credentials to ClientSuccess for auto ingestion of the flat file.

Note: Files can be automatically pulled from the SFTP server 4 to 6 times a day, or whenever the file is successfully updated.