Commit graph

53 commits

Author SHA1 Message Date
Kumi 930dd014fe
feat: add delay for SSH server readiness in CI
Some checks failed
Test! / test (push) Failing after 1m38s
Introduced a step in the CI test workflow to wait for the SSH server to start before proceeding with subsequent actions. This ensures that the server is fully operational and can accept connections before any attempts are made to copy the public key into the Docker container, avoiding potential race conditions where the server might not be ready in time, leading to failed CI runs. The addition of a simple loop with a timeout mechanism improves the reliability of the CI pipeline by making sure tests only run when all services are up and running.
2024-04-22 18:05:50 +02:00
Kumi 78fcec897e
fix(ci): ensure fresh SSH server container setup
Some checks failed
Test! / test (push) Failing after 1m45s
Ensured the SSH server Docker container is always rebuilt from a clean state by forcefully removing any pre-existing container before creation. Additionally, updated the SSH connection user to 'replication' to align with new security protocols.

- The force removal of the existing 'ssh-server' container avoids potential conflicts or inconsistencies due to leftover state from previous test runs, ensuring a more reliable integration testing environment.
- Switching the SSH connection to use the 'replication' user account instead of 'root' enhances security and adheres to best practices by minimizing the use of elevated privileges.

These changes contribute to a more stable and secure CI pipeline by reducing flakiness in tests and aligning with security best practices.
2024-04-22 17:58:32 +02:00
Kumi c72c45f47b
feat(workflows): ensure SSH dir exists before scanning host
Some checks failed
Test! / test (push) Failing after 1m17s
Preemptively creating the `~/.ssh` directory before attempting to add the SSH server's host key to `known_hosts`. This change prevents potential failures in CI workflows where the `.ssh` directory might not exist on a fresh runner environment, ensuring a smoother and more reliable setup for SSH connections.

This adjustment enhances the resilience of the testing workflow by avoiding errors related to missing directories, thereby improving automated testing reliability and efficiency.
2024-04-22 17:06:21 +02:00
Kumi efe8c1a37f
fix: correct Docker build context for SSH server
Some checks failed
Test! / test (push) Failing after 2m30s
The Docker build command for the SSH server container was incorrectly pointing to the Dockerfile instead of the directory containing it. This change corrects the build context to the correct directory, ensuring that all necessary files in `./ci-tests/` are included during the build process. This fix addresses issues with missing dependencies during the container build phase.
2024-04-22 17:01:42 +02:00
Kumi c5ae1eef67
refactor(test.yml): use apt-get without sudo in CI
Some checks failed
Test! / test (push) Failing after 1m18s
Optimized the Docker installation commands in the CI workflow by removing `sudo` when calling `apt-get`. This adjustment caters to environments where the CI runner executes as root or has adequate permissions, streamlining command execution and avoiding potential issues related to `sudo` availability or configuration. This change harmonizes with CI practices that favor minimalist, permission-aware setups.
2024-04-22 16:59:14 +02:00
Kumi d4afc756c4
feat(workflows): Add Docker installation step
Some checks failed
Test! / test (push) Failing after 8s
To facilitate the execution of tests within a controlled environment, Docker is now installed as part of the CI testing workflow. This addition ensures that any Docker-dependent steps, specifically the building and running of an SSH Server Docker container, execute seamlessly. Previously, the assumption was that Docker would be pre-installed on the runner, which could lead to inconsistencies or failures in environments where this was not the case. By explicitly including the Docker installation step, the workflow becomes more portable and resilient across different CI execution environments.
2024-04-22 16:58:23 +02:00
Kumi 7438e0f28b
Some checks failed
Test! / test (push) Failing after 1m1s
2024-04-22 16:55:07 +02:00
Kumi d1bf34ab1f
feat: Introduce automated SSH testing flow
Implemented a new CI pipeline configured to automatically test SSH connectivity as part of the project's testing phase. This involves creating and configuring a Docker container to serve as an SSH server, generating an SSH key pair for secure connection, and ensuring the build environment can establish an SSH connection to the container without manual intervention. The setup aims to automate and streamline testing of SSH-related functionalities, enhancing the reliability of code changes affecting remote server interactions.

- A Dockerfile was added to set up an SSH server in a container.
- The CI pipeline (`test.yml`) was configured to build this container, generate SSH keys, copy the public key to the container, and attempt an SSH connection.

This enhancement supports more robust testing processes, particularly for features that interact with remote servers via SSH.
2024-04-22 16:47:20 +02:00
Kumi e82ccb2701
feat: Enhance stability and configurability
- Extended copyright to reflect the current year.
- Incremented project version to indicate new features and fixes.
- Added a new script entry for easier execution, increasing utility and accessibility.
- Updated project URLs for better alignment with current infrastructure.
- Refactored settings path for simplicity and consistency across deployments.
- Improved code readability and maintenance across several modules by cleaning up redundant code, adding missing type annotations, and ensuring consistent code formatting.
- Enhanced logging capabilities and error handling to improve diagnostics and troubleshooting, supporting more robust error recovery mechanisms.
- Implemented more graceful handling of termination signals to ensure clean shutdown and resource cleanup, enhancing the robustness of the application in production environments.
- Introduced command-line argument parsing for configuration file path customization, improving flexibility in different runtime environments.

These changes collectively improve the project's maintainability, reliability, and user experience, laying a stronger foundation for future development.
2024-04-22 16:39:33 +02:00
Kumi 6df9d7b9dc
Set default name for logger, version bump 2022-09-20 06:55:52 +00:00
Kumi 00d9b4edfd
Remove custom logging, version bump 2022-09-20 05:00:58 +00:00
Kumi 33a80e8884
Version bump 2022-09-20 04:58:51 +00:00
Kumi 8689910f78
Fix missing import for typing, more type hints 2022-09-20 04:54:06 +00:00
Kumi 8c9e3f5944
Version bump 2022-09-19 18:06:19 +00:00
Kumi 5d9a1e87a7
Use relative imports 2022-09-19 18:05:58 +00:00
Kumi d91130fe06
Need to set dbclass before use. 2022-09-19 10:21:01 +00:00
Kumi b5ab22cb6b
Rename init file 2022-09-19 07:34:07 +00:00
Kumi b885eafc1f
File structure for packaging 2022-09-19 07:33:33 +00:00
Kumi d5e560d945
Allow arbitrary database type to be passed in preparation for Academon integration 2022-09-19 07:25:19 +00:00
Kumi b736226140 Prevent re-addition of finished files to queue 2021-12-03 09:08:51 +01:00
Kumi fc56013305 Don't mark ignored files as completed, just ignore them 2021-12-03 08:45:16 +01:00
Kumi 23f08ec9dc Drop wrong argument from checkFileCompletion call 2021-11-30 18:33:19 +01:00
Kumi ce951b0a4e Use unbuffered output for systemd service 2021-11-30 18:15:30 +01:00
Kumi 5db445db9b Fix incorrect path in README 2021-11-30 18:11:03 +01:00
Kumi e5935d0bee Fix location used by assertDirectories 2021-11-30 18:01:52 +01:00
Kumi c5a5500654 Change prints to logs 2021-11-30 17:52:40 +01:00
Kumi 579abf3afc Add socket timeout errors to list of exceptions 2021-11-30 17:48:10 +01:00
Kumi b81ea65b77 Add skipped file to temp list of processed files 2021-11-30 17:43:28 +01:00
Kumi ac4c719480 Remove print statement 2021-11-30 17:35:59 +01:00
Kumi 2d4760f5ed Minor improvement to logging output 2021-11-30 17:33:47 +01:00
Kumi 072047390b Fix Vessel object initialization in fromConfig() 2021-11-30 17:20:12 +01:00
Kumi c072f15b16 Add option to ignore a directory on a vessel 2021-11-30 15:44:50 +01:00
Kumi 4de807b37e Fix value in default settings file, add README 2021-11-30 08:58:29 +01:00
Kumi d0953837ec Bugfixes 2021-11-30 07:25:24 +01:00
Kumi f1094bdbda Implement moving to completed directory 2021-11-29 14:50:16 +01:00
Kumi 82737215fe Survive reboot and directory loss
"log" to stdout
2021-11-26 11:23:39 +01:00
Kumi e62af0f71f Fix currentUpload() return value 2021-11-26 07:52:16 +01:00
Kumi 25844a1f59 Fix listdir call 2021-11-26 07:49:27 +01:00
Kumi 5318285556 Fix SSHClient connect call
Add Port setting to ini
2021-11-26 07:39:21 +01:00
Kumi 1e6d6b38dc Add sqlite3 journal file to .gitignore 2021-11-26 07:06:40 +01:00
Kumi bfc66698de Add credentials to Vessels/Connections 2021-11-26 07:03:13 +01:00
Kumi 5f8a22a5c7 Add comment on getFiles method 2021-11-26 06:51:03 +01:00
Kumi d6a5498fe1 Fix getFiles method on Directory 2021-11-26 06:50:07 +01:00
Kumi 06f6f4ced6 Upload logic finalized 2021-11-25 19:03:58 +01:00
Kumi 4185609289 A few more lines in VesselThread 2021-11-25 17:21:04 +01:00
Kumi b6eb599dd0 Some work on VesselThread 2021-11-25 17:09:40 +01:00
Kumi 7cfcc99909 Logic to add and delete files to processing queue 2021-11-25 17:02:09 +01:00
Kumi f3f6e74c89 Logic improvements
More documentation
2021-11-25 16:31:49 +01:00
Kumi 313c24f727 More documentation strings 2021-11-25 13:42:00 +01:00
Kumi 48bc92653d Fixing Dog Handler usage
Lots of documentation
2021-11-25 10:40:25 +01:00