ftp

Uses an FTP profile to list files on a remote FTP server, read file contents, or upload query results as a file.

Command properties

Syntax

To list files:

To read file contents:

To upload a file:

Options

format={text|csv|tsv|json}

File format (default: text)

• text: Reads or writes one line as one record.
• csv: Reads or writes in CSV format.
• tsv: Reads or writes in tab-separated format.
• json: Reads or writes in JSON format.

offset=LONG

Number of records to skip in the cat operation. Specify a non-negative integer. (default: 0)

limit=LONG

Maximum number of records to read in the cat operation. Specify a non-negative integer.

encoding=STR

File encoding (default: utf-8)

header=BOOL

Whether to use the first row of CSV or TSV files as a header in the cat operation. (default: f)

fields=FIELD,...

List of fields to write in the put operation. Separate with commas (,). If not specified, writes the line field. For JSON format, writes all fields if not specified.

append=BOOL

Whether to append content to an existing file in the put operation. Cannot be used together with the overwrite option. (default: f)

overwrite=BOOL

Whether to overwrite an existing file in the put operation. Cannot be used together with the append option. (default: f)

Target

PROFILE

FTP profile name. Specify the FTP connect profile registered in the system.

PATH

Path to the file or directory on the FTP server. In the cat operation, you can use a wildcard (*) in the file name.

Output fields

Output fields for the ls operation:

Output fields for the cat operation depend on the format option. In text format, a line of text is assigned to the line field. In csv and tsv formats, fields are assigned based on headers or column indices. In json format, JSON key-value pairs are assigned as fields. The _file field containing the original file path is assigned in all formats.

Error codes

Parse errors

Runtime errors

Description

The ftp command communicates with a remote FTP server using an FTP profile registered in the system. Three operations are supported:

• ls: Lists files and directories at the specified path. Directories are listed first, followed by files, sorted alphabetically. The ls operation acts as a driver query.
• cat: Reads the contents of the file at the specified path and converts them into records. You can use a wildcard (*) in the file name. The cat operation acts as a driver query.
• put: Writes input records to a file at the specified path. The put operation acts as a transforming query and passes input records unchanged to the next command. The put operation runs on the Control Node (reducer).

You can use query parameters (${}) in profile names and file paths.

Examples

1. List files on an FTP server

   ftp myprofile ls /data/logs
   

   Lists files in the /data/logs directory using the myprofile FTP profile.

2. Read a text file from an FTP server

   ftp myprofile cat /data/logs/access.log
   

   Reads the /data/logs/access.log file in text format and converts each line into a record.

3. Read a CSV file with a header

   ftp myprofile cat format=csv header=t /data/logs/report.csv
   

   Reads a CSV file using the first row as field names.

4. Read multiple JSON files using a wildcard

   ftp myprofile cat format=json /data/logs/*.json
   

   Reads all files with the .json extension from the /data/logs/ directory.

5. Upload query results to an FTP server as a CSV file

   table duration=1d web_logs
   | stats count by method
   | ftp myprofile put format=csv fields=method,count /data/report.csv
   

   Uploads query results in CSV format to the FTP server.

6. Append content to an existing file

   table duration=1h web_logs
   | ftp myprofile put format=json append=t /data/logs/combined.json
   

   Appends content to an existing file in JSON format.