Skip to content
GitLab
Commit 285caa82 authored by Dillenn Terumalai's avatar Dillenn Terumalai
Browse files

Initial commit

parents
Hide whitespace changes
Inline Side-by-side
.env 0 → 100644
View file @ 285caa82
ID=labo1
SFTP_URL=${ID}@spsp-sftp.vital-it.ch:/data
\ No newline at end of file
.gitignore 0 → 100644
View file @ 285caa82
*
!logs
!outbox
!sent
!.env
!.gitignore
!.pub
!README.md
!spsp
\ No newline at end of file
.pub 0 → 100644
View file @ 285caa82
-----BEGIN PGP PUBLIC KEY BLOCK-----
mQINBF0xzeABEAC7Qcq4NY8bI4TsGIZRNJWOZb8mLlPDwFdVCzyIj8PPsfs4kFc3
1e6xMEfWMO39qACMsta3R2MAsxxbwOjWmiK398pmpZjGX+/RuCH2Frczr9Pl3xFv
+eZJqThlSYh0x1cl+7kgxo1Ac1R/+ixk0PfPdnidCGMMn5GT188l7PfqVSax/Wnx
ChHzGA5svbkrFR3C2OarxP5hcusuOe/5yfZ1p6ZjhV7vdGzLx+6PytpBSzz0alMm
5MoA8GfLMKg7pz7NIaQMjkXGkQpPD99U1VfXSGIjjlPf4eKTmxclCDVotcsDxWtV
SeD2z1X+xAoEtH87XLrjlg9oTT74Dz76rOu+H6O+5kGuVlX3teMNuEeA1qHL9idS
qnV30Q8Lrw/jm1u2WJn1adVaDyuhQElYthHVYMQSvgdelHTMLGhVMg0JuuU9TRFj
7A1DnSitogymC6iSUfl8mJ+P4Q6wqRDKiz+GZH3UBMoSlekkJpInqkaTFgw3Ves8
+7rK92ywGeDQAKUzq2AMBA3nqZ8eWtp9kV+OaYbnQzWbOyOj/WxbQr+BLfDAcq2v
Z1ChJ3us6a2IKNF04FiQ8UmmT6E09DZGcMP5f13anMbPoJ3OVXr/SzSDhl/5Vy7W
JLq/nH67fIX1RVr+s4Wu5o30J3zLaaOaXEJuQjZQNpToUw6oKIb7eghhmQARAQAB
tC9EaWxsZW5uIFRlcnVtYWxhaSA8ZGlsbGVubi50ZXJ1bWFsYWlAc2liLnN3aXNz
PokCVAQTAQgAPhYhBO50rHT5D+yitFrf4q2yPHjqJRjnBQJdMc3gAhsjBQkJZgGA
BQsJCAcCBhUKCQgLAgQWAgMBAh4BAheAAAoJEK2yPHjqJRjnbisP/iM4wYHXLYeL
64hSyL35ltazQEnLQWf3PPyFFLnnZpsp5MoaP5V4TRzzlOPNdQji3HBYxyHD7Dop
HZHJetLAeRatRthvgYj8xyKPE5vWY+K7QUIUXC5oETYGa1j/Tn+PpB10XMKRmg8O
2rPF1+3yYPVjkF6cwcz5tg3aE1FTeedYdmkinVGx64Y6X/LHX1EuxG5Hlstg8SYe
ptsDkmXKOOdpCEOudDjA0Qqzft8oQhdZMKJ5GqA01lIwvgvLOeEyx/4GkAL0zZXS
8vErXRRmF10kmOtnrvymp8VvLHInlxtNWbIzKyDE2uHeHlOfyX6DQEvlVqHqkbu2
dfg7B9l1wh3pbvgXwMsi74TNLCVKwaroh5jJ9NEjgJPeS6o5Q4ZNP401Ek+de7T3
XzshUWoc9wQ9GfHRIXWUcdy0wJbK9vcgn7+jhMI+Ou+jN81ALkvE4cwGHDBVLlJx
2oG15JKOZjx7zABqSXIesI5s6LOrJRlfo2dVYu+id7VInCSjrwxpAh+TBUtA1Ist
VDhEj6xzA4kfo7FgPeh4lNzE1O/21YPOuwZ9Wz5OLtXFg5la5CzG7wYBXGWZ6zii
BkDYOobQz72zcYJfYa1NEUGG5knrrJgwrl3lJzPaIta3zzfx9APU6EkyFhYD0H4L
nYo2HTz6LIg9CxGeKLsMiX0ARgOp0gYiuQINBF0xzeABEAC8zf2DRrL1bYVRzjMt
XsUtkSQQ3B7EeqHVXzo2laX15LtYYPucsoxVA/zgbriTZJ2MY8v1KpGhFtJ9M/JI
u6nDoSrIkRfiZpldRFZYJCgzOsvcjttmgHPczx9PKpmQsr8Xkb/9uFJSuUz057LX
S4gg02/o1DiAF6rWTSGIgRC/3/ig8DBXy9100ZrYLHLZ9EkPDYpat3dh8tBUpYiC
szOkRyWzRJNHDVjLE+u05DbOrLcr2CphNYmD+PJib+XgZyVdno4v+FwMO1RnzJr4
R31UCdfdWl8wnBfnSZeemNhRftELtObST9P6s3QuzjyIhKsuU24uHes3fOcTe70I
iwPZ30R0+mODMav3DAL5bL4LZCX82rRqVRKooEBJwTkzgCsx9nLzgi5bFuxFlani
3hU7tctT3GDnOsiUs3BlLmSeXUfLbLh7QB36OfguWAfmbc45gmkoVfSD3SsoiNq+
N2lLU4PV64IiI+pP9zB8Z8QCjN6a3aTHvbcVLKkwsr2hgIY+81zDmQ96gN1TeNUA
uCJus21Vb36wxRiXp/KQ9qFppHwIK4aF/YX5iAEEwogUZHwjd57Rbnu5AzeVmQEx
/Az/BGdEvvqfcpy/OonnbSRFrr1meWWS9H5gC6HqU71F5bAhVtP1kTFB+r6UdREi
tCS9Sw/5zLmQyauoUUl3Rg9a2wARAQABiQI8BBgBCAAmFiEE7nSsdPkP7KK0Wt/i
rbI8eOolGOcFAl0xzeACGwwFCQlmAYAACgkQrbI8eOolGOdHlhAAqRy28pwZ+BOf
LZykrEJr0sIF4U4a5uxcIxAW9/4KjeL0G/s4o+jXtAwDzBHb23hgfrjRcJF23b8G
G3R3XWNiLHS2q+ztq6EVv1IZIPbipqVjB43gLtdQMuHc2tB0BeI8qB52A+8szuzY
gkyRXhl+Dxm3Fh3PN9Mvr3sUJibP/ffkE5f6Kq6RC3Adgq0EL+6Bn0P8p3IRE5tm
qFt7pNK+3u77ZoN/YsWsmJPeuNvdIJrssbquaZFFV7mCIXu4iAASGhVbc40Aw1Y5
RjtyRo+AdedtUm4XbZ08WxxIw6M8VkrvCEXECWj7TVTEcEBf8ZXxdwyiWWrf7n/z
rs4Pk2c9xjZk0j9pVN4SW0DpkD4Mqc+hMI6pJN9OG3CBLPa97y23ms3gk4Y/Y0FE
6vDT69Wkn+NpjQCGhla+/7s8lemuj5ZTggJnua96ppPOv6FV0RkjBkfdsmPQxUux
fK88XOBLccMQm9oUyo8LfsFV2acrVKTeRoaF+c6cps8t6GN7ywRYOXsNn9j0kC3p
4vFfv6MSExNRSqlSqkb9U523aCl+LXxH5bB0k34aBMMcPT7QwplUTz0kycKjzZYu
rkejnSlarEizLwaAgXVPQdo7OLocGhPBxtRyk1pMoh9zMWTDl5/hIKVuT/gwAlnx
5v9P/E7LjReU1Tbo9pOb1Weg4MD5GK4=
=AQco
-----END PGP PUBLIC KEY BLOCK-----
README.md 0 → 100644
View file @ 285caa82
# SPSP Transfer-tool
This is the GitLab repo of the official transfer-tool for SPSP.
## Table of Contents
- [Getting Started](#getting-started)
- [Upload the SSH public key](#upload-the-SSH-public-key)
- [Installation](#installation)
- [Use the transfer-tool](#use-the-transfer-tool)
- [Use the automatic mode](#use-the-automatic-mode)
- [Authors](#authors)
## Getting Started
These instructions will allow you to use locally the tool to transfer your data.
### Prerequisites
If you want to be able to use the tool smoothly, make sure that you have:
- OS: macOS or Linux - This script can only run on those operative systems
- SSH: public key uploaded - This script is assuming that you have generated and transferred your public SSH key to the SPSP SFTP Server. If not, please read the [Installation](#installation) chapter
- GPG: gpg avaiable - This script uses GPG to encrypt the data, make sure that you can run this command (`which gpg`)
- (Optional)CRON: automatic mode - If you want to activate the automatic mode of the transfer-tool make sure that you can setup a CRON task
## Installation
A step by step series of commands that tell you how to setup properly the transfer tool.
Start by cloning the transfer-tool on your local machine:
```bash
git clone git@gitlab.sib.swiss:SPSP/transfer-tool.git
```
## Authors
* **Dillenn TERUMALAI** - *Initial work* - [dillenn.terumalai@sib.swiss](mailto:dillenn.teruamlai@sib.swiss)
logs/.gitignore 0 → 100644
View file @ 285caa82
*
!.gitignore
\ No newline at end of file
outbox/.gitignore 0 → 100644
View file @ 285caa82
*
!.gitignore
\ No newline at end of file
sent/.gitignore 0 → 100644
View file @ 285caa82
*
!.gitignore
\ No newline at end of file
spsp 0 → 100755
View file @ 285caa82
#!/usr/bin/env bash
# SPSP transfer-tool is developped and maintained by SIB. For more informations, please contact spsp-support@sib.swiss
VERSION="1.0.0" # bumped to support spsp --help, spsp --version
# Define colors that are used in the help screen
ESC_SEQ="\x1b["
COL_RESET=${ESC_SEQ}"39;49;00m"
COL_LYELLOW=${ESC_SEQ}"33;01m"
COL_LGREEN=${ESC_SEQ}"32;01m"
COL_CYAN=${ESC_SEQ}"0;36m"
COL_GREEN=${ESC_SEQ}"0;32m"
COL_MAGENTA=${ESC_SEQ}"0;35m"
# Define the UID of the recipient for GPG encryption
RECIPIENT="dillenn.terumalai@sib.swiss"
# Check for the machine compatibility
UNAMEOUT="$(uname -s)"
case "${UNAMEOUT}" in
Linux*) MACHINE=linux;;
Darwin*) MACHINE=mac;;
*) MACHINE="UNKNOWN"
esac
if [ "$MACHINE" == "UNKNOWN" ]; then
echo "Unsupported system type"
echo "System must be a Macintosh or Linux"
echo ""
echo "System detection determined via uname command"
echo "If the following is empty, could not find uname command: $(which uname)"
echo "Your reported uname is: $(uname -s)"
fi
# Set environment variables for dev
if [ "$MACHINE" == "linux" ]; then
SEDCMD="sed -i"
SHACMD="sha256sum"
GPGCMD="gpg"
elif [ "$MACHINE" == "mac" ]; then
SEDCMD="sed -i .bak"
SHACMD="shasum -a 256"
GPGCMD="gpg"
fi
showVersion() {
intro="\n🔐${COL_GREEN} SPSP transfer-tool${COL_RESET}\n"
intro="$intro ${COL_CYAN}Version ${VERSION}\n${COL_RESET}"
printf "$intro"
}
showHelp() {
showVersion
usage="${COL_LYELLOW}Usage:\n${COL_RESET}"
usage="$usage ./spsp <cmd> <options>"
commands="${COL_LYELLOW}Commands:\n${COL_RESET}"
commands="$commands compress <folder> Compress the ${COL_LGREEN}FASTQ${COL_RESET} files to ${COL_LYELLOW}tar.gz${COL_RESET} file\n"
commands="$commands encrypt <compressed file> Encrypt a ${COL_LGREEN}tar.gz${COL_RESET} file using a public key\n"
commands="$commands transfer <encrypted file> Transfer the encrypted file using ${COL_LGREEN}sftp${COL_RESET}\n"
commands="$commands auto Automatically run the commands in a correct order to ${COL_CYAN}compress${COL_RESET}, ${COL_GREEN}encrypt${COL_RESET} & ${COL_MAGENTA}transfer${COL_RESET}\n"
commands="$commands help Shows Help screen\n"
commands="$commands logs Displays the content of the main log \n"
options="${COL_LYELLOW}Options:\n${COL_RESET}"
options="$options --help, -h Shows Help (this screen)\n"
options="$options --version, -V, version Show Version\n"
examples="${COL_LYELLOW}Examples:\n${COL_RESET}"
examples="$examples ${COL_CYAN}$ ./script compress myfolder${COL_RESET}\n"
examples="$examples ${COL_CYAN}$ ./script encrypt myfile.tar.gz${COL_RESET}\n"
examples="$examples ${COL_CYAN}$ ./script transfer myencryptedfile${COL_RESET}\n"
examples="$examples ${COL_CYAN}$ ./script auto${COL_RESET}\n"
examples="$examples ${COL_CYAN}$ ./script logs${COL_RESET}\n"
printf "\n"
printf "$usage\n\n"
printf "$commands\n"
printf "$options\n"
printf "$examples\n"
}
compressFolder() {
message="Compressing folder ${COL_LGREEN}$1${COL_RESET}..."
printf "$message\n"
tar -czvf $1.tar.gz $1
message="Folder is compressed as ${COL_LYELLOW}$1.tar.gz${COL_RESET}!"
printf "$message\n"
}
transferFile() {
for f in "$1"; do
printf "progress\nput $f" | sftp -b- "${SFTP_URL}" || exit 1
done
}
signFile() {
message="Hashing the file ${COL_LGREEN}$1${COL_RESET} using SHA-256..."
printf "$message\n"
$SHACMD $1 > $1.sha256
message="Hash has been properly generated for ${COL_LYELLOW}$1${COL_RESET}!"
printf "$message\n"
}
encryptFile() {
message="Encrypting file ${COL_LGREEN}$1${COL_RESET}..."
printf "$message\n"
$GPGCMD -o $1.gpg -e -r $RECIPIENT $1
message="File is encrypted as ${COL_LYELLOW}$1.gpg${COL_RESET}!"
printf "$message\n"
}
deleteSource() {
message="Deleting the source ${COL_LGREEN}$1${COL_RESET}..."
printf "$message\n"
rm -rf $1
message="${COL_LYELLOW}$1${COL_RESET} has been properly deleted!"
printf "$message\n"
}
# If we pass any arguments...
if [ $# -gt 0 ]; then
if [ -f .env ]; then
source .env
else
echo "No .env file found within current working directory ${pwd}"
echo "You need to create this file to use this script"
echo "Please refer to the documentation for further informations"
exit 1
fi
if [ "$1" == "--version" ] || [ "$1" == "-v" ] || [ "$1" == "version" ]; then
showVersion
exit 1
fi
if [ "$1" == "--help" ] || [ "$1" == "-H" ] || [ "$1" == "help" ]; then
showHelp
exit 1
fi
if [ "$1" == "init" ]; then
echo "SPSP: Initializing transfer-tool..."
if [ ! -x "$(command -v tar)" ]; then
echo "Error: tar command is not available."
exit 0
fi
if [ ! -x "$(command -v sftp)" ]; then
echo "Error: sftp command is not available."
exit 0
fi
if [ ! -f .env ]; then
echo "No .env file found within current working directory $(pwd)"
echo "Create a .env file before re-initializing"
exit 0
fi
if [ ! -f spsp ]; then
echo "No spsp file found within current working directory $(pwd)"
exit 0
fi
echo "SPSP: Making spsp transfer-tool command available"
chmod +x spsp
echo "SPSP: Importing SPSP public key for encryption protocol..."
$GPGCMD --import .pub
echo ""
echo "SPSP: Complete!"
echo "SPSP: You can now use SPSP transfer-tool"
echo "SPSP: List all the commands with:"
echo "./spsp help"
# Edit .env file to set SFTP_URL and PROFILE
elif [ "$1" == "auto" ]; then
CURRENTDATE=`date +"%Y-%m-%d"`
echo "SPSP: Starting the transfer-tool..."
OUTBOX=outbox
if [ ! -d "$OUTBOX" ]; then
echo "No $OUTBOX folder found within current working directory ${pwd}"
echo "Creating the $OUTBOX folder..."
mkdir $OUTBOX
fi
SENT=sent
if [ ! -d "$SENT" ]; then
echo "No $SENT folder found within current working directory ${pwd}"
echo "Creating the $SENT folder..."
mkdir $SENT
fi
LOGS=logs
if [ ! -d "$LOGS" ]; then
echo "No $LOGS folder found within current working directory ${pwd}"
echo "Creating the $LOGS folder..."
mkdir $LOGS
fi
LOGFILE="./logs/spsp-$CURRENTDATE.log"
if [ ! -f $LOGFILE ]; then
touch $LOGFILE
fi
for DIRECTORY in *; do
if [ -d "$DIRECTORY" ] && [ $DIRECTORY != $OUTBOX ] && [ $DIRECTORY != $SENT ] && [ $DIRECTORY != $LOGS ]; then
if [ -f $DIRECTORY/*.fastq ] && [ -f $DIRECTORY/*.xlsx ]; then
echo $DIRECTORY
else
echo "[`date +"%T"`] local.ERROR: Missing files (*.fastq or *.xlsx) inside ${DIRECTORY}" >> $LOGFILE
echo "SPSP: Error detected please check $LOGFILE"
exit 0
fi
fi
done
echo ""
echo "SPSP: Transfer completed!"
# Compress a folder given as an input
elif [ "$1" == "compress" ]; then
shift 1
compressFolder $@
elif [ "$1" == "transfer" ]; then
shift 1
transferFile $@
elif [ "$1" == "sign" ]; then
shift 1
signFile $@
elif [ "$1" == "encrypt" ]; then
shift 1
encryptFile $@
else
message="Command does not exist, please check the help\n ${COL_LGREEN}/script.sh help${COL_RESET}"
printf "$message\n"
fi
fi
\ No newline at end of file
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment