【运维】Nginx网关与热部署

Nginx简介

Nginx是一个轻量级的Http服务器,Nginx包含一个单一的master进程和多个worker进程。所有这些进程都是单线程,并且设计为同时处理成千上万个连接。Nginx使用操作系统事件机制来快速响应这些请求。
Nginx的master进程负责读取配置文件、处理套接字、派生worker进程、打开日志文件和编译嵌入式的Perl脚本。
Nginx的worker进程运行在一个忙碌的事件循环处理中,用于处理进入的连接。每一个Nginx模块被构筑在worker中,因此任何请求处理、过滤、处理代理的连接和更多的操作都在worker进程中完成。

1
2
3
4
5
6
7
# docker 中运行的nginx
root 3699 3675 0 2018 ? 00:00:00 nginx: master process /opt/gitlab/embedded/sbin/nginx -p /var/opt/gitlab/nginx
systemd+ 3770 3699 0 2018 ? 00:16:35 nginx: worker process
systemd+ 3771 3699 0 2018 ? 00:17:19 nginx: worker process
systemd+ 3772 3699 0 2018 ? 00:15:19 nginx: worker process
systemd+ 3773 3699 0 2018 ? 00:24:05 nginx: worker process
systemd+ 3774 3699 0 2018 ? 00:01:45 nginx: cache manager process

Nginx 容器实例

为了方便扩展和迁移,公司的nginx都跑在Docker环境中。Docker版的nginx可以一键迁移,自动重启,相当于进程守护。
nginx官方镜像不支持ssl,目前的浏览器都把非https的网站标识为不安全,因此支持https是非常必要的,本文在marvambass/nginx-ssl-secure镜像的基础上做了部分修改,以满足需要,镜像下载地址为:https://hub.docker.com/r/cbbing/nginx-ssl-secure

实践

docker-compose.yml文件

下面为最常用的nginx运行的docker配置,对外开放80,443端口,即http和https。

1
2
3
4
5
6
7
8
9
10
11
12
version: '2.1'
services:
nginx:
image: cbbing/nginx-ssl-secure
volumes:
- ./conf/conf.d/cer:/etc/nginx/external
- ./conf/conf.d:/etc/nginx/conf.d
- /mydata/logs/nginx_logs:/var/log/nginx
ports:
- 80:80
- 443:443
restart: always

说明:
读取conf配置,映射到宿主机的conf/conf.d文件夹
读取cer证书,映射到宿主机的conf/conf.d/cer文件夹
日志文件映射到宿主机的/mydata/logs/nginx_logs
restart设置为always,相当于进程守护,宕掉后自动重启。

conf配置

平时打交道最多的就是conf文件,热部署。
一个典型的conf文件如下:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
upstream xxx_api{
server 10.105.10.1:992 ; #api03
}

server {
listen 80;
server_name api.xxx.com;
client_max_body_size 50M;
location / {
add_header X-Cache-Status $upstream_cache_status;
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_connect_timeout 500s;
proxy_read_timeout 500s;
proxy_send_timeout 500s;
proxy_pass http://xxx_api;
}

access_log /var/log/nginx/xxx_api_http.log main;
}

server {
listen 443;
server_name api.xxx.com;
ssl on;
ssl_certificate conf.d/cer/1_api.xxx.com_bundle.crt;
ssl_certificate_key conf.d/cer/2_api.xxx.com.key;
ssl_session_timeout 5m;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2; #按照这个协议配置
ssl_ciphers ECDHE-RSA-AES128-GCM-SHA256:HIGH:!aNULL:!MD5:!RC4:!DHE;#按照这个套件配置
ssl_prefer_server_ciphers on;
client_max_body_size 50M;
location / {
add_header X-Cache-Status $upstream_cache_status;
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_connect_timeout 500s;
proxy_read_timeout 500s;
proxy_send_timeout 500s;
proxy_pass http://xxx_api;
}

access_log /var/log/nginx/xxx_api_https.log main;
}

第一部分upstream,可以定义多个后端服务器,通过IPHash/最小连接/轮询/加权轮询/主备等策略实现负载均衡。
第二部分server,是域名访问的配置项,包括端口,具体域名,location中转设置项
第三部分log,是日志存储的路径。日志的格式在nginx.conf中定义的。

1
2
3
log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
'$status [$request_body] $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';

也可以只定义一个443端口,80端口通过如下方式重定义:

1
2
3
4
5
6
server
{
listen 80;
server_name api.xxx.com;
rewrite ^/(.*) https://api.xxx.com/$1 permanent;
}

API网关

一套成熟的API网关包括如下几个方面:

API管理包括:

  • 定义和发布
  • 安全
  • 流量控制
  • 持续监控与维护
  • API分析

负载均衡

1
2
3
4
5
6
7
upstream api{
least_conn;
server 10.105.19.5:891 ;
server 10.105.19.5:892 ;
server 10.105.19.5:893 ;
server 10.105.19.5:894 ;
}

这里运行了4个api服务,通过最小连接策略调用。

1
2
3
4
5
[root@VM_196_57_centos ~]# docker ps
2c8792e403bf "/bin/sh -c 'gunic..." 3 days ago Up 3 days 0.0.0.0:891->8000/tcp p891_api_1
2c8792e403ba "/bin/sh -c 'gunic..." 3 days ago Up 3 days 0.0.0.0:892->8000/tcp p892_api_1
2c8792e403bb "/bin/sh -c 'gunic..." 3 days ago Up 3 days 0.0.0.0:893->8000/tcp p893_api_1
2c8792e403bc "/bin/sh -c 'gunic..." 3 days ago Up 3 days 0.0.0.0:894->8000/tcp p894_api_1

静态网页服务器

静态文件配置demo.conf

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
server
{
listen 80;
server_name www.xxx.com;

location / {
root /home/demo;
index login.html;
}
access_log /var/log/nginx/demo_http.log main;
}

server {
listen 443;
server_name www.xxx.com;
ssl on;
ssl_certificate conf.d/cer/1_www.xxx.com_bundle.crt;
ssl_certificate_key conf.d/cer/2_www.xxx.com.key;
ssl_session_timeout 5m;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2; #按照这个协议配置
ssl_ciphers ECDHE-RSA-AES128-GCM-SHA256:HIGH:!aNULL:!MD5:!RC4:!DHE;#按照这个套件配置
ssl_prefer_server_ciphers on;
client_max_body_size 50M;
location / {
root /home/demo;
index login.html;
}
access_log /var/log/nginx/demo_https.log main;
}

这个配置即以/home/demo目录为静态文件目录,不过/home/demo是容器内的路径,需要映射到宿主机目录/mydata/demo,实现动态更新。

1
2
3
4
5
6
7
8
9
10
11
12
13
version: '2.1'
services:
nginx:
image: cbbing/nginx-ssl-secure
volumes:
- ./conf/conf.d/cer:/etc/nginx/external
- ./conf/conf.d:/etc/nginx/conf.d
- /mydata/logs/nginx_logs:/var/log/nginx
- /mydata/demo:/home/demo

ports:
- 80:80
- 443:443

热部署

更新了conf文件后,执行nginx -t,先排查语法错误,通过后执行nginx -s reload重新加载配置文件实现热部署。

nginx热部署的方式时,等旧的worker执行完成后kill掉,以更新后的配置启动新的worker

1
2
3
4
5
6
7
[root@VM_231_16_centos ~]# docker exec -it nginxnew_nginx_1 nginx -t
nginx: [emerg] duplicate upstream "myweb" in /etc/nginx/conf.d/power.xxx.conf:3
nginx: configuration file /etc/nginx/nginx.conf test failed
[root@VM_231_16_centos ~]# docker exec -it nginxnew_nginx_1 nginx -t
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful
[root@VM_231_16_centos ~]# docker exec -it nginxnew_nginx_1 nginx -s reload

参考

https://www.jianshu.com/p/5eab0f83e3b4
What Is API Management?
https://cloud.tencent.com/document/product/628/11755
https://cloud.tencent.com/developer/article/1149103
https://zhuanlan.zhihu.com/p/34943332
https://www.infoq.cn/article/comparing-api-gateway-performances

Author: Binger Chen
Link: http://www.kekefund.com/2019/05/09/nginx-reload/
Copyright Notice: All articles in this blog are licensed under CC BY-NC-SA 4.0 unless stating additionally.