Sftp

NAME:
   singularity storage create sftp - SSH/SFTP

USAGE:
   singularity storage create sftp [command options]

DESCRIPTION:
   --host
      SSH host to connect to.
      
      E.g. "example.com".

   --user
      SSH username.

   --port
      SSH port number.

   --pass
      SSH password, leave blank to use ssh-agent.

   --key-pem
      Raw PEM-encoded private key.
      
      If specified, will override key_file parameter.

   --key-file
      Path to PEM-encoded private key file.
      
      Leave blank or set key-use-agent to use ssh-agent.
      
      Leading `~` will be expanded in the file name as will environment variables such as `${RCLONE_CONFIG_DIR}`.

   --key-file-pass
      The passphrase to decrypt the PEM-encoded private key file.
      
      Only PEM encrypted key files (old OpenSSH format) are supported. Encrypted keys
      in the new OpenSSH format can't be used.

   --pubkey-file
      Optional path to public key file.
      
      Set this if you have a signed certificate you want to use for authentication.
      
      Leading `~` will be expanded in the file name as will environment variables such as `${RCLONE_CONFIG_DIR}`.

   --known-hosts-file
      Optional path to known_hosts file.
      
      Set this value to enable server host key validation.
      
      Leading `~` will be expanded in the file name as will environment variables such as `${RCLONE_CONFIG_DIR}`.

      Examples:
         | ~/.ssh/known_hosts | Use OpenSSH's known_hosts file.

   --key-use-agent
      When set forces the usage of the ssh-agent.
      
      When key-file is also set, the ".pub" file of the specified key-file is read and only the associated key is
      requested from the ssh-agent. This allows to avoid `Too many authentication failures for *username*` errors
      when the ssh-agent contains many keys.

   --use-insecure-cipher
      Enable the use of insecure ciphers and key exchange methods.
      
      This enables the use of the following insecure ciphers and key exchange methods:
      
      - aes128-cbc
      - aes192-cbc
      - aes256-cbc
      - 3des-cbc
      - diffie-hellman-group-exchange-sha256
      - diffie-hellman-group-exchange-sha1
      
      Those algorithms are insecure and may allow plaintext data to be recovered by an attacker.
      
      This must be false if you use either ciphers or key_exchange advanced options.
      

      Examples:
         | false | Use default Cipher list.
         | true  | Enables the use of the aes128-cbc cipher and diffie-hellman-group-exchange-sha256, diffie-hellman-group-exchange-sha1 key exchange.

   --disable-hashcheck
      Disable the execution of SSH commands to determine if remote file hashing is available.
      
      Leave blank or set to false to enable hashing (recommended), set to true to disable hashing.

   --ask-password
      Allow asking for SFTP password when needed.
      
      If this is set and no password is supplied then rclone will:
      - ask for a password
      - not contact the ssh agent
      

   --path-override
      Override path used by SSH shell commands.
      
      This allows checksum calculation when SFTP and SSH paths are
      different. This issue affects among others Synology NAS boxes.
      
      E.g. if shared folders can be found in directories representing volumes:
      
          rclone sync /home/local/directory remote:/directory --sftp-path-override /volume2/directory
      
      E.g. if home directory can be found in a shared folder called "home":
      
          rclone sync /home/local/directory remote:/home/directory --sftp-path-override /volume1/homes/USER/directory

   --set-modtime
      Set the modified time on the remote if set.

   --shell-type
      The type of SSH shell on remote server, if any.
      
      Leave blank for autodetect.

      Examples:
         | none       | No shell access
         | unix       | Unix shell
         | powershell | PowerShell
         | cmd        | Windows Command Prompt

   --md5sum-command
      The command used to read md5 hashes.
      
      Leave blank for autodetect.

   --sha1sum-command
      The command used to read sha1 hashes.
      
      Leave blank for autodetect.

   --skip-links
      Set to skip any symlinks and any other non regular files.

   --subsystem
      Specifies the SSH2 subsystem on the remote host.

   --server-command
      Specifies the path or command to run a sftp server on the remote host.
      
      The subsystem option is ignored when server_command is defined.

   --use-fstat
      If set use fstat instead of stat.
      
      Some servers limit the amount of open files and calling Stat after opening
      the file will throw an error from the server. Setting this flag will call
      Fstat instead of Stat which is called on an already open file handle.
      
      It has been found that this helps with IBM Sterling SFTP servers which have
      "extractability" level set to 1 which means only 1 file can be opened at
      any given time.
      

   --disable-concurrent-reads
      If set don't use concurrent reads.
      
      Normally concurrent reads are safe to use and not using them will
      degrade performance, so this option is disabled by default.
      
      Some servers limit the amount number of times a file can be
      downloaded. Using concurrent reads can trigger this limit, so if you
      have a server which returns
      
          Failed to copy: file does not exist
      
      Then you may need to enable this flag.
      
      If concurrent reads are disabled, the use_fstat option is ignored.
      

   --disable-concurrent-writes
      If set don't use concurrent writes.
      
      Normally rclone uses concurrent writes to upload files. This improves
      the performance greatly, especially for distant servers.
      
      This option disables concurrent writes should that be necessary.
      

   --idle-timeout
      Max time before closing idle connections.
      
      If no connections have been returned to the connection pool in the time
      given, rclone will empty the connection pool.
      
      Set to 0 to keep connections indefinitely.
      

   --chunk-size
      Upload and download chunk size.
      
      This controls the maximum size of payload in SFTP protocol packets.
      The RFC limits this to 32768 bytes (32k), which is the default. However,
      a lot of servers support larger sizes, typically limited to a maximum
      total package size of 256k, and setting it larger will increase transfer
      speed dramatically on high latency links. This includes OpenSSH, and,
      for example, using the value of 255k works well, leaving plenty of room
      for overhead while still being within a total packet size of 256k.
      
      Make sure to test thoroughly before using a value higher than 32k,
      and only use it if you always connect to the same server or after
      sufficiently broad testing. If you get errors such as
      "failed to send packet payload: EOF", lots of "connection lost",
      or "corrupted on transfer", when copying a larger file, try lowering
      the value. The server run by [rclone serve sftp](/commands/rclone_serve_sftp)
      sends packets with standard 32k maximum payload so you must not
      set a different chunk_size when downloading files, but it accepts
      packets up to the 256k total size, so for uploads the chunk_size
      can be set as for the OpenSSH example above.
      

   --concurrency
      The maximum number of outstanding requests for one file
      
      This controls the maximum number of outstanding requests for one file.
      Increasing it will increase throughput on high latency links at the
      cost of using more memory.
      

   --set-env
      Environment variables to pass to sftp and commands
      
      Set environment variables in the form:
      
          VAR=value
      
      to be passed to the sftp client and to any commands run (eg md5sum).
      
      Pass multiple variables space separated, eg
      
          VAR1=value VAR2=value
      
      and pass variables with spaces in in quotes, eg
      
          "VAR3=value with space" "VAR4=value with space" VAR5=nospacehere
      
      

   --ciphers
      Space separated list of ciphers to be used for session encryption, ordered by preference.
      
      At least one must match with server configuration. This can be checked for example using ssh -Q cipher.
      
      This must not be set if use_insecure_cipher is true.
      
      Example:
      
          aes128-ctr aes192-ctr aes256-ctr aes128-gcm@openssh.com aes256-gcm@openssh.com
      

   --key-exchange
      Space separated list of key exchange algorithms, ordered by preference.
      
      At least one must match with server configuration. This can be checked for example using ssh -Q kex.
      
      This must not be set if use_insecure_cipher is true.
      
      Example:
      
          sntrup761x25519-sha512@openssh.com curve25519-sha256 curve25519-sha256@libssh.org ecdh-sha2-nistp256
      

   --macs
      Space separated list of MACs (message authentication code) algorithms, ordered by preference.
      
      At least one must match with server configuration. This can be checked for example using ssh -Q mac.
      
      Example:
      
          umac-64-etm@openssh.com umac-128-etm@openssh.com hmac-sha2-256-etm@openssh.com
      


OPTIONS:
   --disable-hashcheck    Disable the execution of SSH commands to determine if remote file hashing is available. (default: false) [$DISABLE_HASHCHECK]
   --help, -h             show help
   --host value           SSH host to connect to. [$HOST]
   --key-file value       Path to PEM-encoded private key file. [$KEY_FILE]
   --key-file-pass value  The passphrase to decrypt the PEM-encoded private key file. [$KEY_FILE_PASS]
   --key-pem value        Raw PEM-encoded private key. [$KEY_PEM]
   --key-use-agent        When set forces the usage of the ssh-agent. (default: false) [$KEY_USE_AGENT]
   --pass value           SSH password, leave blank to use ssh-agent. [$PASS]
   --port value           SSH port number. (default: 22) [$PORT]
   --pubkey-file value    Optional path to public key file. [$PUBKEY_FILE]
   --use-insecure-cipher  Enable the use of insecure ciphers and key exchange methods. (default: false) [$USE_INSECURE_CIPHER]
   --user value           SSH username. (default: "$USER") [$USER]

   Advanced

   --ask-password               Allow asking for SFTP password when needed. (default: false) [$ASK_PASSWORD]
   --chunk-size value           Upload and download chunk size. (default: "32Ki") [$CHUNK_SIZE]
   --ciphers value              Space separated list of ciphers to be used for session encryption, ordered by preference. [$CIPHERS]
   --concurrency value          The maximum number of outstanding requests for one file (default: 64) [$CONCURRENCY]
   --disable-concurrent-reads   If set don't use concurrent reads. (default: false) [$DISABLE_CONCURRENT_READS]
   --disable-concurrent-writes  If set don't use concurrent writes. (default: false) [$DISABLE_CONCURRENT_WRITES]
   --idle-timeout value         Max time before closing idle connections. (default: "1m0s") [$IDLE_TIMEOUT]
   --key-exchange value         Space separated list of key exchange algorithms, ordered by preference. [$KEY_EXCHANGE]
   --known-hosts-file value     Optional path to known_hosts file. [$KNOWN_HOSTS_FILE]
   --macs value                 Space separated list of MACs (message authentication code) algorithms, ordered by preference. [$MACS]
   --md5sum-command value       The command used to read md5 hashes. [$MD5SUM_COMMAND]
   --path-override value        Override path used by SSH shell commands. [$PATH_OVERRIDE]
   --server-command value       Specifies the path or command to run a sftp server on the remote host. [$SERVER_COMMAND]
   --set-env value              Environment variables to pass to sftp and commands [$SET_ENV]
   --set-modtime                Set the modified time on the remote if set. (default: true) [$SET_MODTIME]
   --sha1sum-command value      The command used to read sha1 hashes. [$SHA1SUM_COMMAND]
   --shell-type value           The type of SSH shell on remote server, if any. [$SHELL_TYPE]
   --skip-links                 Set to skip any symlinks and any other non regular files. (default: false) [$SKIP_LINKS]
   --subsystem value            Specifies the SSH2 subsystem on the remote host. (default: "sftp") [$SUBSYSTEM]
   --use-fstat                  If set use fstat instead of stat. (default: false) [$USE_FSTAT]

   Client Config

   --client-ca-cert value                           Path to CA certificate used to verify servers
   --client-cert value                              Path to Client SSL certificate (PEM) for mutual TLS auth
   --client-connect-timeout value                   HTTP Client Connect timeout (default: 1m0s)
   --client-expect-continue-timeout value           Timeout when using expect / 100-continue in HTTP (default: 1s)
   --client-header value [ --client-header value ]  Set HTTP header for all transactions (i.e. key=value)
   --client-insecure-skip-verify                    Do not verify the server SSL certificate (insecure) (default: false)
   --client-key value                               Path to Client SSL private key (PEM) for mutual TLS auth
   --client-no-gzip                                 Don't set Accept-Encoding: gzip (default: false)
   --client-scan-concurrency value                  Max number of concurrent listing requests when scanning data source (default: 1)
   --client-timeout value                           IO idle timeout (default: 5m0s)
   --client-use-server-mod-time                     Use server modified time if possible (default: false)
   --client-user-agent value                        Set the user-agent to a specified string (default: rclone/v1.62.2-DEV)

   General

   --name value  Name of the storage (default: Auto generated)
   --path value  Path of the storage

   Retry Strategy

   --client-low-level-retries value  Maximum number of retries for low-level client errors (default: 10)
   --client-retry-backoff value      The constant delay backoff for retrying IO read errors (default: 1s)
   --client-retry-backoff-exp value  The exponential delay backoff for retrying IO read errors (default: 1.0)
   --client-retry-delay value        The initial delay before retrying IO read errors (default: 1s)
   --client-retry-max value          Max number of retries for IO read errors (default: 10)
   --client-skip-inaccessible        Skip inaccessible files when opening (default: false)

Last updated