关于 一月, 2014 的文章

使用Tengine替代Nginx作为负载均衡服务器

Tengine是由淘宝网发起的Web服务器项目。它在Nginx的基础上,针对大访问量网站的需求,添加了很多高级功能和特性。Tengine的性能和稳定性已经在大型的网站如淘宝网,天猫商城等得到了很好的检验。它的最终目标是打造一个高效、稳定、安全、易用的Web平台。

而本文章中的配置参数,都已经在生产环境中得到应用,针对10万高并发的状态以及8核CPU做了相应的优化。

1. 增大Nginx用户的open files数值
[root@idc1-server1 ~]$ sudo -i
[root@idc1-server1 ~]# vim /etc/security/limits.conf

 
nginx hard nofile 102400
nginx soft nofile 102400

2. 优化内核参数
[root@idc1-server1 ~]# vim /etc/sysctl.conf

 
# For Nginx
net.ipv4.ip_forward = 0
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.default.accept_source_route = 0
kernel.sysrq = 0
kernel.core_uses_pid = 1
net.ipv4.tcp_syncookies = 1
kernel.msgmnb = 65536
kernel.msgmax = 65536
kernel.shmmax = 68719476736
kernel.shmall = 4294967296
net.ipv4.tcp_max_tw_buckets = 6000
net.ipv4.tcp_sack = 1
net.ipv4.tcp_window_scaling = 1
net.ipv4.tcp_rmem = 4096 87380 4194304
net.ipv4.tcp_wmem = 4096 16384 4194304
net.core.wmem_default = 8388608
net.core.rmem_default = 8388608
net.core.rmem_max = 16777216
net.core.wmem_max = 16777216
net.core.netdev_max_backlog = 262144
net.core.somaxconn = 262144
net.ipv4.tcp_max_orphans = 3276800
net.ipv4.tcp_max_syn_backlog = 262144
net.ipv4.tcp_timestamps = 0
net.ipv4.tcp_synack_retries = 1
net.ipv4.tcp_syn_retries = 1
net.ipv4.tcp_tw_recycle = 1
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_mem = 94500000 915000000 927000000
net.ipv4.tcp_fin_timeout = 1
net.ipv4.tcp_keepalive_time = 30
net.ipv4.ip_local_port_range = 1024 65000
fs.file-max = 102400

[root@idc1-server1 ~]# sysctl -p

3. 安装并配置Tengine
[root@idc1-server1 ~]# /etc/init.d/nginx stop
[root@idc1-server1 ~]# yum erase nginx
[root@idc1-server1 ~]# useradd -M -g nginx -d /opt/tengine -s /sbin/nologin -c "Nginx web server" nginx

[root@idc1-server1 ~]# mkdir -p /root/dong/downloads/
[root@idc1-server1 ~]# cd /root/dong/downloads/
[root@idc1-server1 downloads]# yum install gcc gcc-c+= pcre-devel openssl openssl-devel
[root@idc1-server1 downloads]# wget http://tengine.taobao.org/download/tengine-2.0.0.tar.gz
[root@idc1-server1 downloads]# tar xzvf tengine-2.0.0.tar.gz
[root@idc1-server1 downloads]# cd tengine-2.0.0
[root@idc1-server1 tengine-2.0.0]# ./configure --prefix=/opt/tengine
[root@idc1-server1 tengine-2.0.0]# make
[root@idc1-server1 tengine-2.0.0]# make install
阅读全文 »

,

2 Comments

快速使用proxychains设置代理上网

背景介绍:
由于公司内部上网权限管理比较严格,因此放置在公司内部的测试服务器只有少数可以上网。
而proxychains就可以非常简单快速的设置好代理,让其它不能上网的服务器通过能上网的服务器进行上网。
proxychains只需要在不能上网的机器上安装好,而能够上网的服务器,是不需要任何配置的,只要能SSH登陆就可以。

服务器环境:
可以上网:192.168.92.2
不能上网:192.168.92.138

配置步骤:
下载proxychains
[dong.guo@192.168.92.2 ~]$ git clone https://github.com/haad/proxychains
[dong.guo@192.168.92.2 ~]$ tar czf proxychains.tgz proxychains/
[dong.guo@192.168.92.2 ~]$ scp proxychains.tgz dong.guo@192.168.92.138:/home/dong.guo/

安装proxychains
[dong.guo@192.168.92.138 ~]$ tar xzf proxychains.tgz
[dong.guo@192.168.92.138 ~]$ cd proxychains
[dong.guo@192.168.92.138 proxychains]$ ./configure --sysconfdir=/usr/local/etc
[dong.guo@192.168.92.138 proxychains]$ make
[dong.guo@192.168.92.138 proxychains]$ sudo make install
[dong.guo@192.168.92.138 proxychains]$ sudo cp src/proxychains.conf /usr/local/etc/

[dong.guo@192.168.92.138 proxychains]$ ssh -NfD 9050 dong.guo@192.168.92.2

通过proxychains执行命令,即可通过192.168.92.2的网络了
[dong.guo@192.168.92.138 proxychains]$ cd
[dong.guo@192.168.92.138 ~]$ curl http://ifconfig.me
curl: (6) Couldn't resolve host 'ifconfig.me'
[dong.guo@192.168.92.138 ~]$ /usr/local/bin/proxychains4 curl http://ifconfig.me
[proxychains] config file found: /usr/local/etc/proxychains.conf
[proxychains] preloading /usr/local/lib/libproxychains4.so
[proxychains] DLL init
[proxychains] Strict chain ... 127.0.0.1:9050 ... ifconfig.me:80 ... OK
110.184.227.147

除了curl,执行yum等命令的时候,都只要在前面加上/usr/local/bin/proxychains4,就都可以通过192.168.92.2的网络了。
这种方式可以在仅仅需要的时候使用,而不改变任何服务器的网络配置,非常不错。

1 Comment

绕开SST通过IST方式添加Node到Percona XtraDB Cluster

参考资料:
https://github.com/percona/xtradb-cluster-tutorial/blob/master/instructions/Avoiding%20SST.rst#bad-configuration

服务器:
idc1-server1, idc1-server2, idc1-server3

1. 安装Percona XtraDB Cluster,在 idc1-server1,idc1-server2,idc1-server3 上
$ sudo yum install http://www.percona.com/downloads/percona-release/percona-release-0.0-1.x86_64.rpm
$ sudo yum install http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm
$ sudo yum install Percona-Server-shared-compat
$ sudo yum install Percona-XtraDB-Cluster-server Percona-XtraDB-Cluster-client
$ sudo mkdir -p /opt/mysql/{data,tmp,run,binlogs,log}
$ sudo chown mysql:mysql /opt/mysql/{data,tmp,run,binlogs,log}
$ sudo -i
# su - mysql
$ mysql_install_db --user=mysql --datadir=/opt/mysql/data/
$ exit
# exit

2. 配置 my.cnf,在 idc1-server1 上
[heydevops@idc1-server1 ~]$ sudo vim /etc/my.cnf

 
[mysqld]
# basic settings
datadir = /opt/mysql/data
tmpdir = /opt/mysql/tmp
socket = /opt/mysql/run/mysqld.sock
pid-file = /opt/mysql/run/mysqld.pid

# innodb settings
default-storage-engine = INNODB
innodb_file_per_table = 1
log-bin = /opt/mysql/binlogs/bin-log-mysqld
log-bin-index = /opt/mysql/binlogs/bin-log-mysqld.index
innodb_data_home_dir = /opt/mysql/data
innodb_data_file_path = ibdata1:10M:autoextend
innodb_log_group_home_dir = /opt/mysql/data

# xtradb cluster settings
binlog_format = ROW
wsrep_cluster_name = mycluster
wsrep_cluster_address = gcomm://10.100.1.3,10.100.1.4,10.100.1.5
wsrep_node_address = 10.100.1.3
wsrep_provider = /usr/lib64/libgalera_smm.so
wsrep_sst_method = xtrabackup
wsrep_sst_auth = sst:secret
wsrep_provider_options = "gcache.size=8G;"
wsrep_sst_receive_address = 10.100.1.3
innodb_locks_unsafe_for_binlog = 1
innodb_autoinc_lock_mode = 2

# server id
server-id=123

# other settings
[mysqld_safe]
log-error = /opt/mysql/log/mysqld.log
pid-file = /opt/mysql/run/mysqld.pid
open-files-limit = 8192

[mysqlhotcopy]
interactive-timeout

[client]
port = 3306
socket = /opt/mysql/run/mysqld.sock
default-character-set = utf8

阅读全文 »

, , , ,

2 Comments

在Linux上安装配置BitTorrent Sync

背景介绍:
目前我们线上的前端服务器数量比较多,超过200多台,每次发布新应用的时候,都是将软件包放在一台专门的Push服务器上,再由所有的前端服务器通过rsync自动同步。但随着前端服务器的数量越来越多,Push服务器的带宽已经成为了瓶颈。
而BitTorrent Sync这种P2P方式的同步则是一种解决方案。同时它的跨平台支持也非常好,无论是Windows,Linux,Mac OS,甚至手机端都有相应的客户端。虽然目前尚未开源,但可以免费使用,还是很不错的。

下面,就是我们在线上的纯Linux测试环境中的安装与配置步骤:
Servers:
idc2-server1,idc2-server2,idc2-server3

1. 下载BitTorrent Sync,在所有服务器上:
$ sudo wget http://download-lb.utorrent.com/endpoint/btsync/os/linux-x64/track/stable -O /tmp/btsync_x64.tar.gz
$ sudo mkdir /opt/btsync
$ cd /opt/btsync
$ sudo tar xzf /tmp/btsync_x64.tar.gz

2. 创建服务管理脚本,在所有服务器上:
$ sudo vim /etc/init.d/btsync

 
#!/bin/sh
#
# description: starts and stops the btsync client

CONF=/opt/btsync/btsync.cfg
PROC=/opt/btsync/btsync
PIDFILE=/opt/btsync/btsync.pid

start() {
  PID1=$(pidof btsync)
  if [ -z ${PID1} ]; then
    echo -n "Starting BitTorrent Sync: "
    ${PROC} --config ${CONF}
  else
    echo "BitTorrent Sync is already running at pid:${PID1}"
  fi
  return $?
}  

stop() {
  echo -n "Stopping BitTorrent Sync: "
  PID1=$(pidof btsync)
  if [ ! -z ${PID1} ]; then
    kill -9 ${PID1}
    echo "OK"
  else
    echo "Failed"
  fi
  return $?
}  

status() {
  PID1=$(pidof btsync)
  PID2=$(cat ${PIDFILE}) 
  echo -n "Checking BitTorrent Sync: "
  if [ ! -z ${PID1} ] && [ "${PID1}" -eq "${PID2}" ]; then
    echo "OK"
  else
    echo "Failed"
  fi
  return $?
}  

case "$1" in
  start)
   start
  ;;
  stop)
    stop
  ;;
  restart)
    stop
    sleep 1
    start
  ;;
  status)
    status
  ;;
  *)
    echo $"Usage: $0 {start|stop|restart|status}"
    exit 2
esac

$ sudo chmod +x /etc/init.d/btsync

3. 创建用于同步的目录,在所有服务器上:
$ sudo mkdir /opt/btsync_transfer

4. 创建配置文件,在idc2-server1上:
[heydevops@idc2-server1 btsync]$ sudo vim /opt/btsync/btsync.cfg

 
{ 
  "device_name": "idc2-server1",
  "listening_port" : 8889, // 0 - randomize port

  "check_for_updates" : false,
  "use_upnp" : false,

  "storage_path" : "/opt/btsync",
  "pid_file" : "/opt/btsync/btsync.pid",

  "download_limit" : 0, // 0 - no limit
  "upload_limit" : 0, 

  "webui" :
  {
    "listen" : "0.0.0.0:8888",
    "login" : "admin",
    "password" : "btsync"
  }

  ,
  "folder_rescan_interval" : 60,
  "lan_encrypt_data" : false,
  "lan_use_tcp" : true
}

5. 创建同步所需的密钥,在idc2-server1上:
$ sudo /etc/init.d/btsync start

打开Web UI:http://idc2-server1:8888
用户名: admin
密码: btsync

点击 "Add Folder",
在 "Path" 中输入 "/opt/btsync_transfer"
点击 "Generate" 得到 "Secret" 为 "ALUORWDEWOLV354ZHPHFT4TSQO67JWQAN"
如下图所示:
2
阅读全文 »

,

6 Comments

在Python中使用Dict来传递参数

今天在帮Wianm修改一个Python脚本的时候,学到了一个有趣的东西,那就是如何在Python中使用Dict来传递参数。
在此之前我只会用{0} .format() 和 %s 的方式,将要传递的变量一个个的写在后面。

代码内容:

 
family = {"dad": "Dong", "mum": "Hong", "kid": "Yun"}
print "Dad:%(dad)s Mum:%(mum)s Kid:%(kid)s" % family 
print "Dad:%s Mum:%s Kid:%s" % tuple(family.values())

打印结果:

 
Dad:Dong Mum:Hong Kid:Yun
Dad:Dong Mum:Hong Kid:Yun

3 Comments