This project is implemented by members from the Intelligence Fusion Research Center (IFRC), Zhejiang University, and GenTel.io.
👋👋👋 Welcome to follow our project!
Please feel free to email us at mhan@zju.edu.cn if you have any questions.
If you find this work useful, please feel free to leave a star⭐️!
SafeLLMDeploy is a project designed to ensure the secure deployment of large language model (LLM) services. It provides a comprehensive set of automated scripts and configuration files that implement essential security measures during the deployment process. The project includes features like firewall configuration, access control, and the secure setup of services such as Nginx and Fail2Ban. These components work together to protect the LLM service from potential security threats and ensure stable, secure operation. Users can easily deploy the system through a straightforward installation process, and customize security rules to suit their needs, including defining whitelisted IP addresses and ports. Additionally, SafeLLMDeploy automates the process of installing system packages and required services, and ensuring proper service startup and security configurations.
The project contains the following key directories and files:
scripts/install.sh: Handle the entire installation process.start.sh: Start all security services.stop.sh: Stop all security services.configure_nginx.sh: Install and configure Nginx.configure_fail2ban.sh: Install and configure Fail2Ban.configure_nftables.sh: Configure nftables rules.nginx/check.bash: Check if the port configurations in the Nginx configuration file are correct.generate_api_key.sh: Generate or update API key for Nginx.ollama.conf: Configure Nginx for access control, routing, and security restrictions.nftables/whitelist_ips.txt: Whitelist file listing allowed IP addresses.whitelist_ports.txt: Whitelist file listing allowed ports.rules.nft: Manage how network traffic is filtered and processed.fail2ban/jail.local, ollama-api.conf: Configuration files for Fail2Ban.
- Make sure you have sufficient permissions (typically root or use sudo) before running the installation script.
- Modify the whitelist files (whitelist_ips.txt and whitelist_ports.txt) according to your needs to allow specific IP addresses and ports to access the services.
1. Clone the project to your local machine:
git clone git@github.com:IFRC-ZJU-Edu/SafeLLMDeploy.git
cd SafeLLMDeploy2. Run the installation script:
chmod +x scripts/install.sh
./scripts/install.sh