Skip to main content

MetaLocator SFTP import job

Importing data via secure FTP

Written by Michael Fatica

SFTP Data Import Setup

To enable automated, background SFTP imports on a recurring basis, you must contact the MetaLocator Help Desk. This functionality requires both the Backgrounding add-on and an Enterprise subscription.


Before You Begin

  • Provide the Help Desk with your public SSH key.
    MetaLocator will generate and share your SFTP credentials.

  • Submit the IP address or IP range that should be whitelisted for SFTP access.

  • Ensure your system uploads the data file to the SFTP server on a regular schedule.

  • Confirm your private key is in RSA (PKCS#1) format:

    -----BEGIN RSA PRIVATE KEY-----
    (key content)
    -----END RSA PRIVATE KEY-----
  • Verify that your file is properly formatted and ready for direct import.


Initial Import Setup (Required)

Before configuring the SFTP job, you must first establish your import settings:

  1. Go to Data → Import → CSV Upload.

  2. Upload your file manually.

  3. Configure:

    • Column mappings

    • Import options (e.g., Update Existing & Insert New)

  4. Save the import configuration.

This step ensures the system can reuse your mapping rules during automated imports.


Creating the SFTP Import Job

  1. Navigate to Data → Import → SFTP.

  2. Enter the following details:

    • SFTP Username

    • Remote Path (e.g., /writable/locator-*.csv)

    • File Name (file imported in CSV step above)

    • RSA Private Key (PEM format)

    Example:

    -----BEGIN RSA PRIVATE KEY-----
    MIIEpQIBAAKCAQEA3Tz2mr7SZiAMfQyuvBjM9OiZ1BjP5CE/Wm/Rr500PRK+Lh9x
    -----END RSA PRIVATE KEY-----
  3. Click Test SFTP Credentials to confirm connectivity.

    • If the test fails, verify the username, file path, and key format.

  4. Enable “Keep SFTP Data in Sync”, then proceed.

  5. Review and confirm:

    • Field mappings

    • Import options (recommended: Update Existing & Insert New)

  6. Complete the setup.


Finalizing & Scheduling

  1. Open the Background Jobs panel (top-right icon).

  2. Select your SFTP job.

  3. Click Publish and Schedule.

  4. Run a forced execution to validate:

    • Successful import

    • Email notification

    • Entry in Import History


File Naming & Upload Requirements

  • Upload files as UTF-8 encoded CSVs.

  • Use the recommended naming convention:

    locator-YYYYMMDD-HHMM.csv

    This ensures the system can identify the most recent file.

  • Upload files daily to the provided SFTP location.

  • Do not use the SFTP server for storage or backups.

  • The system retains up to 30 files for a maximum of 120 days.


Important Configuration Notes

Local File Name

  • This determines which saved import mapping is applied.

  • It must match the filename used during your initial manual import.

  • Use a unique, consistent name (e.g., 12639-locator.csv) to avoid conflicts.


SFTP Settings (Provided by Help Desk)

  • SFTP Username – Login credential

  • SFTP Remote Path – Full file path (supports wildcards)

  • Local File Name – Mapping reference filename

  • Email Notify List – Semicolon-separated email recipients

  • RSA Private Key – Authentication key (PEM format)


Optional Job Parameters

  • sftp_latest_only (1/0)
    Downloads only the most recent file (used with wildcards).

  • sftp_skip_data_combinator (1/0)
    Skips custom import plugins when enabled.

  • sftp_once_only (1/0)
    Prevents reprocessing of previously downloaded files (do not use with static filenames).

  • sftp_delete_after_download (1/0)
    Deletes the file from the server after successful import.
    If multiple files match, only the newest is processed; older ones are removed.


Best Practices

  • Always confirm you can download the most recent file before enabling automation.

  • Keep file naming consistent to avoid mapping issues.

  • Run a manual test before scheduling recurring jobs.

Did this answer your question?