|
|
|
# Transfer Tool Documentation
|
|
|
|
|
|
|
|
The aime of the Transfer Tool (TT) is to provide a tool allowing users of the Swiss Pathogen Surveillance Platform to easily and securely transfer sequencing files. The TT is a simple shell script relying on multiple libraries (surch as GPG and OpenSSH) to archive, hash, encrypt and transfer FASTQ files with their metadata.
|
|
|
|
|
|
|
|
To understand how the TT works, it helps to look at a diagram. In the example on the right, Labo1 wants to send DNA sequences of bacteria/viruses in a protected manner. To do so, the data exchange will rely on asymetric/public key encryption. The SPSP server has a public key and a private key, which are two mathematically related encryption keys. The public key can be shared with any laboratory, but only the SPSP server has the private key.
|
|
|
|
|
|
|
|
First, Lab1 uses the TT to compress the sequences files (*.fastq) and metadata file (*.xlsx) into a tar.gz archive.
|
|
|
|
Then, the TT generates a unique hash of the previously generated archive using SHA-256 algorithm, meaning that if the content changes even slightly, the hash will be completely different.
|
|
|
|
After that, the TT uses SPSP’s public key to encrypt the archive, turning it into something scrambled.
|
|
|
|
Finally, the encrypted archive and the hash are uploaded using SFTP protocol which runs over the SSH protocol (which provides communication security and strong encryption).
|
|
|
|
|
|
|
|
On the server side, once the archive is properly uploaded, the server will decrypt the encrypted archive using its own private key. Then, it will compare the uploaded hash to the hash generated by the server from the decrypted archive to ensure that the content was never changed during the transfer. Finally, if everything goes well, the metadata file is parsed and loaded inside the database. As the server cannot guess for which project of the laboratory the uploaded sequences belong to, an assignment task will be generated.
|
|
|
|
|
|
|
|
Advantages of the Transfer Tool and SFTP services:
|
|
|
|
**It keeps the data safe from hackers.** Asymetric/public key encryption means only the laboratory sender and the SPSP server have access to the unencrypted data. While in transit, the data remains completely encrypted.
|
|
|
|
**It uses a secured tunnel.** SFTP protocol protects the integrity of the data using encryption and cryptographic hash functions, and authenticates both the server and the laboratory.
|
|
|
|
**It checks that the data is left untouched.** By generating a hash, the TT makes sure that the data was never modified during the whole process.
|
|
|
|
|
|
|
|
# Table of Contents
|
|
|
|
|
|
|
|
- [How does the Transfer Tool (TT) work?](how-does-the-transfer-tool-tt-work)
|
|
|
|
- [Getting Started](getting-started)
|
|
|
|
- [Set up a shared drive between users of the same SPSP group](set-up-a-shared-drive-between-users-of-the-same-spsp-group)
|
|
|
|
- [Upload the SSH public key](upload-the-SSH-public-key)
|
|
|
|
- [Installation](installation)
|
|
|
|
- [Use the Transfer Tool](use-the-transfer-tool)
|
|
|
|
- [How to prepare FASTQ files with the metadata file](how-to-prepare-fastq-files-with-the-metadata-file)
|
|
|
|
- [How to transfer files easily](how-to-transfer-files-easily)
|
|
|
|
- [Use the automatic mode in combination with a CRON task](use-the-automatic-mode-in-combination-with-a-cron-task)
|
|
|
|
- [Debugging](debugging)
|
|
|
|
- [Authors](authors) |
|
|
\ No newline at end of file |
