Data Partners (Batch)

You must first create an account with GroundTruth in order to proceed with the integration.
If you do not have an account, please contact your account manager, or send an email to: [email protected]

Overview

This document outlines how a partner can send user location signals to GroundTruth for interpretation of user location data. A partner can provide this information via uploading a file with multiple records on a daily basis.

Feasibility

Before integration, please communicate to our account manager / business development team member with an estimate number of rows per day to assess feasibility.

Data preparation

We have strict guidelines for certain fields that will allow our data pipelines to process all your data with minimum issues. The data will be sent over CSV files, so some basic understanding is required.

Mobile Applications Only
GroundTrust is not interested in any data from website or mobile website. So please make sure that all location data sent through this process are from mobile applications.

CSV Headers
The first line must contain the header. You must include all mandatory fields as well as any additional fields you can find in the addendum at the bottom of this document. Please make sure all the data and headers are aligned like a spreadsheet e.g. put date data under the date column.

Data Encoding
If there is no data for a given field on a given record please leave it empty, i.e. no space between the commas (,,).
Values in the file should be CSV escaped (a backslash before the comma).
Data Freshness
The exptimestamp of each record (row) must be within 96 hours of the time of upload.

Limited Country Support
We only support the following countries: US and CA. If you have additional data please inform us so we can explicitly enabled them.

Note: The country field is deprecated. Country is derived using the coordinates.

GDPR
We are adding two new fields to allow data vendors to pass along the status of the user as well as their consent to their data sharing. The two fields are gdpr and gdpr_consent. If the user is from the EU/EEA you must set gdpr to 1, otherwise set to 0. When from EU/EEA gdpr_consent must also have a value, if not keep the field blank, but gdpr_consent is still mandatory.

Mandatory Fields

Each file at a minimum must contain the following fields (can accept additional fields):

Supported Fields

File preparation
Data must be delivered in CSV format and can be compressed. If the data is huge, you must chunk them into parts:

If transferred uncompressed, files must each be 1GB in size
If transferred compressed, files must each be 500MB in size
If the total volume of data transferred is less than this amount, must transfer a single file containing all of the data
We support both .gz and .zip compressed files.

You can also choose to break the payload into multiple files called parts. For example you can break a 400MB file into 2x 200MB files part_0001.csv and part_0002.csv.

Data Uploading

You will be provided a destination folder on our Amazon S3 bucket from which you will be able to upload your data file(s) along with a success file.

S3://dataprovidersapi/<partner_name>

Upload your files in the correct subfolder structure of the following format:

/<year>/<month>/<day>/

For example if today was April 12, 2018 you will need to upload your files in the following subfolder:

S3://dataprovidersapi/your-biz-name/2018/04/12

Once you upload all your files you will need to make sure you also upload a success file named _SUCCESS. The contents of that file should contain the total number of rows transferred to that bucket for that day. There should only be one success file per day. The reason this is needed are:

• to know when all your files are fully uploaded and
• as a checksum to confirm the read was successful