Archive for category Linux&Unix
How to access a blocked web via SSH Tunnel
Posted by mcsrainbow in Linux&Unix on March 10, 2017
Many engineers need to work from home via the VPN, but for the servers which created or deleted by autoscale programs, it’s not that easy to update the VPN routes for them immediately.
So it could be a good idea to access them via the SSH tunnels on a whitelisted host.
I wrote a shell script for that:
https://github.com/mcsrainbow/shell-scripts/blob/master/scripts/create_tunnel.sh
How to install newer version of git on CentOS
Posted by mcsrainbow in Linux&Unix on March 4, 2017
cd /etc/yum.repos.d
vim wandisco-git.repo
[WANdisco-git]
name=WANdisco Distribution of git
baseurl=http://opensource.wandisco.com/rhel/$releasever/git/$basearch
enabled=1
gpgcheck=0
yum install git
rpm -qa | grep git
git-2.11.1-1.WANdisco.346.x86_64
Troubleshooting huge MySQL temp files
Posted by mcsrainbow in Database, Linux&Unix on August 5, 2016
Tonight I got an alert about the disk usage on db1.idc1, its root partition usage was higher than 96% and was increasing.
[root@db1.idc1 ~]# df -hP
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/mylvm-root 102G 92G 4.5G 96% /
tmpfs 32G 0 32G 0% /dev/shm
/dev/sda1 240M 30M 198M 13% /boot
/dev/mapper/mylvm2-var 1.1T 152G 893G 15% /var
After a quick troubleshoot, I found some MySQL temp files took most spaces.
[root@db1.idc1 ~]# cd /tmp
[root@db1.idc1 tmp]# ls -lh | grep G
total 78G
-rw-rw---- 1 mysql mysql 32G Aug 5 07:28 #sql_4862_0.MYD
-rw-rw---- 1 mysql mysql 7.5G Aug 5 07:28 #sql_4862_0.MYI
-rw-rw---- 1 mysql mysql 32G Aug 5 07:28 #sql_4862_2.MYD
-rw-rw---- 1 mysql mysql 7.5G Aug 5 07:28 #sql_4862_2.MYI
[root@db1.idc1 tmp]# lsof | grep /tmp/#sql_4862_0.MYD
mysqld 18530 mysql 282u REG 253,0 33339280800 2883602 /tmp/#sql_4862_0.MYD
[root@db1.idc1 tmp]# ps aux | grep 18530
root 16365 0.0 0.0 103304 916 pts/0 S+ 07:30 0:00 grep --colour=auto 18530
mysql 18530 66.5 4.2 15718552 2795960 ? Sl May20 73373:06 /usr/sbin/mysqld --basedir=/usr --datadir=/var/lib/mysql/ --plugin-dir=/usr/lib64/mysql/plugin --user=mysql --log-error=/var/lib/mysql/db1.idc1.err --pid-file=/var/lib/mysql/db1.idc1.pid --socket=/var/lib/mysql/mysql.sock --port=3306
Then found that two slow queries were running on db1.idc1.
And those two queries were selecting data from 52320599850 == 52 billion rows, from the host 10.8.8.88.
[root@db1.idc1 tmp]# mysql -uusername -ppassword -e 'show full processlist\G'
...
*************************** 10. row ***************************
Id: 34770751
User: username
Host: 10.8.8.88:50657
db: dbname
Command: Query
Time: 303040
State: Sending data
Info: Select ...
Rows_sent: 0
Rows_examined: 52320599850
Rows_read: 52320599850
*************************** 11. row ***************************
Id: 34824847
User: username
Host: 10.8.8.88:50829
db: dbname
Command: Query
Time: 302476
State: Sending data
Info: Select ...
Rows_sent: 0
Rows_examined: 52240222853
Rows_read: 52240222853
...
I killed those two queries then saved the db1.idc1, asked the developers to improve the queries.
[root@db1.idc1 tmp]# mysql -uusername -ppassword
mysql> kill 34770751;
Query OK, 0 rows affected (0.00 sec)
mysql> kill 34824847;
Query OK, 0 rows affected (0.00 sec)
mysql> exit
Bye
[root@db1.idc1 tmp]# df -hP
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/mylvm-root 102G 12G 85G 12% /
tmpfs 32G 0 32G 0% /dev/shm
/dev/sda1 240M 30M 198M 13% /boot
/dev/mapper/mylvm2-var 1.1T 152G 892G 15% /var
Zabbix practices
Posted by mcsrainbow in Linux&Unix on June 28, 2016
Some useful informations about Zabbix server in production.
1. Zabbix server summary:
Hosts: 2,000+
Items: 170,000+ (After disabled 50% non-important items from 300,000+ items)
Zabbix Server: 1
Zabbix Proxy: 3 (For other 3 data centers, total backend hosts < 500)
MySQL Data size: 200G(with TokuDB engine and keep history data in 30 days)
MySQL QPS: 10,000~15,000
2. Zabbix server hardware:
CPU: Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz * 24 Cores
Memory: 64G RAM
Hard drives: 300G SAS*2(for OS)
Data drives: Shannon Direct-IO SSD G2(for MySQL)
3. Zabbix Server optimization:
MySQL: Percona TokuDB engine + Split big tables about history data
Items: Over 95% items are active/trapper
pfSense sucks
Posted by mcsrainbow in Linux&Unix, Network on October 28, 2015
Never use pfSense as the Gateway in production environment, sometimes you have to reboot the whole OS to fix some issues, but the reboot means serious downtime especially when there are some VPN connections between different colocations.
Just buy some professional firewall network devices…
how to create RPM and YUM repository
Posted by mcsrainbow in Linux&Unix on April 21, 2015
Reference:
http://www.ibm.com/developerworks/cn/linux/management/package/rpm/part1/
http://www.ibm.com/developerworks/cn/linux/management/package/rpm/part2/
Goal:
Make a RPM package for compiled nginx 1.1.9 with pcre 8.30 and zlib 1.2.6
Crate a private repository to manage the RPM packages
1. Create a softlink for my home
$ cd /home
$ sudo ln -s dong dong.guo
2. Install some compilation tools
$ sudo yum install gcc gcc-c++ autoconf automake zlib libxml ncurses-devel libgcrypt libtool
3. Install the RPM build tools
$ sudo yum install rpm-build rpmdevtools
4. Configure the RPM build environment
$ cd /home/dong.guo
$ mkdir rpmbuild
$ cd rpmbuild
$ mkdir -pv {BUILD,BUILDROOT,RPMS,SOURCES,SPECS,SRPMS,TMP}
$ cd /home/dong.guo
$ sudo vim .rpmmacros
$ sudo cp .rpmmacros /root/
%_topdir /home/dong.guo/rpmbuild %_tmppath /home/dong.guo/rpmbuild/TMP
5. Download the source packages for nginx-1.1.9
$ cd /home/dong.guo/rpmbuild/SOURCES
$ wget ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-8.30.tar.gz
$ wget http://sourceforge.net/projects/libpng/files/zlib/1.2.6/zlib-1.2.6.tar.gz/download
$ wget http://nginx.org/download/nginx-1.1.9.tar.gz
$ sudo vim nginx
#!/bin/sh
#
# nginx - this script starts and stops the nginx daemin
#
# chkconfig: - 85 15
# description: Nginx is an HTTP(S) server, HTTP(S) reverse \
# proxy and IMAP/POP3 proxy server
# processname: nginx
# config: /usr/local/nginx/conf/nginx.conf
# pidfile: /usr/local/nginx/logs/nginx.pid
# Source function library.
. /etc/rc.d/init.d/functions
# Source networking configuration.
. /etc/sysconfig/network
# Check that networking is up.
[ "$NETWORKING" = "no" ] && exit 0
nginx="/opt/nginx/sbin/nginx"
prog=$(basename $nginx)
NGINX_CONF_FILE="/opt/nginx/conf/nginx.conf"
lockfile=/opt/nginx/lock/nginx.lock
start() {
[ -x $nginx ] || exit 5
[ -f $NGINX_CONF_FILE ] || exit 6
echo -n $"Starting $prog: "
daemon $nginx -c $NGINX_CONF_FILE
retval=$?
echo
[ $retval -eq 0 ] && touch $lockfile
return $retval
}
stop() {
echo -n $"Stopping $prog: "
killproc $prog -QUIT
retval=$?
echo
[ $retval -eq 0 ] && rm -f $lockfile
return $retval
}
restart() {
configtest || return $?
stop
start
}
reload() {
configtest || return $?
echo -n $"Reloading $prog: "
killproc $nginx -HUP
RETVAL=$?
echo
}
force_reload() {
restart
}
configtest() {
$nginx -t -c $NGINX_CONF_FILE
}
rh_status() {
status $prog
}
rh_status_q() {
rh_status >/dev/null 2>&1
}
case "$1" in
start)
rh_status_q && exit 0
$1
;;
stop)
rh_status_q || exit 0
$1
;;
restart|configtest)
$1
;;
reload)
rh_status_q || exit 7
$1
;;
force-reload)
force_reload
;;
status)
rh_status
;;
condrestart|try-restart)
rh_status_q || exit 0
;;
*)
echo $"Usage: $0 {start|stop|status|restart|condrestart|try-restart|reload|force-reload|configtest}"
exit 2
esac
$ sudo chmod +x nginx
$ tree
├── BUILD ├── BUILDROOT ├── RPMS ├── SOURCES │ ├── nginx │ ├── nginx-1.1.9.tar.gz │ ├── pcre-8.30.tar.gz │ └── zlib-1.2.6.tar.gz ├── SPECS ├── SRPMS └── TMP
6. Configure the SPEC file
$ cd /home/dong.guo/rpmbuild/SPECS
$ vim heylinux-nginx-1.1.9.spec
Name: heylinux-nginx
Version: 1.1.9
Release: el6
Summary: Compiled nginx 1.1.9 with pcre 8.30 and zlib 1.2.6
Group: System Environment/Daemons
License: GPL
URL: http://www.drawbrid.ge
Source0: nginx-1.1.9.tar.gz
Source1: pcre-8.30.tar.gz
Source2: zlib-1.2.6.tar.gz
Vendor: guosuiyu@gmail.com
BuildRequires: gcc gcc-c++ autoconf automake openssl-devel
%description
This RPM package is made for compiled nginx 1.1.9, pcre 8.30 and zlib 1.2.6 are not installed,
they just provide the packages for nginx build process.
%prep
tar xzvf $RPM_SOURCE_DIR/pcre-8.30.tar.gz
tar xzvf $RPM_SOURCE_DIR/zlib-1.2.6.tar.gz
tar xzvf $RPM_SOURCE_DIR/nginx-1.1.9.tar.gz
%build
cd nginx-1.1.9
./configure --prefix=/opt/nginx \
--user=nginx \
--group=nginx \
--pid-path=/opt/nginx/run/nginx.pid \
--lock-path=/opt/nginx/lock/nginx.lock \
--with-http_ssl_module \
--with-http_dav_module \
--with-http_flv_module \
--with-http_realip_module \
--with-http_gzip_static_module \
--with-http_stub_status_module \
--with-mail \
--with-mail_ssl_module \
--with-pcre=../pcre-8.30 \
--with-zlib=../zlib-1.2.6 \
--with-debug \
--http-client-body-temp-path=/opt/nginx/tmp/client \
--http-proxy-temp-path=/opt/nginx/tmp/proxy \
--http-fastcgi-temp-path=/opt/nginx/tmp/fastcgi \
--http-uwsgi-temp-path=/opt/nginx/tmp/uwsgi \
--http-scgi-temp-path=/opt/nginx/tmp/scgi
%install
rm -rf $RPM_BUILD_ROOT
cd nginx-1.1.9/
mkdir -p $RPM_BUILD_ROOT/opt/nginx/tmp
mkdir -p $RPM_BUILD_ROOT/opt/nginx/run
mkdir -p $RPM_BUILD_ROOT/opt/nginx/lock
make install DESTDIR=%{buildroot}
mkdir -p $RPM_BUILD_ROOT/etc/init.d
cp -p $RPM_SOURCE_DIR/nginx $RPM_BUILD_ROOT/etc/init.d/
%clean
rm -rf $RPM_BUILD_DIR/pcre-8.30
rm -rf $RPM_BUILD_DIR/zlib-1.2.6
rm -rf $RPM_BUILD_DIR/nginx-1.1.9
%preun
%postun
userdel nginx
%files
/opt/nginx
/etc/init.d/nginx
%post
useradd -M nginx -s /sbin/nologin
%changelog
7. Build the RPM package
$ sudo rpmbuild -bb heylinux-nginx-1.1.9.spec
8. Found the RPM package
$ cd /home/dong.guo/rpmbuild/RPMS/x86_64/
$ ls
heylinux-nginx-1.1.9-el6.x86_64.rpm
9. Copy the RPM package to another server
$ scp -i /home/dong.guo/.ssh/id_rsa heylinux-nginx-1.1.9-el6.x86_64.rpm dong.guo@172.16.1.21:/home/dong/
10. Login the server 172.16.1.21 and install the RPM package
$ sudo rpm -ivh heylinux-nginx-1.1.9-el6.x86_64.rpm
Preparing... ########################################### [100%] 1:heylinux-nginx ########################################### [100%]
$ sudo /etc/init.d/nginx start
Starting nginx: [ OK ]
$ sudo netstat -lntp | grep 80
tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 13581/nginx
11. Crate a private YUM repository
$ sudo yum install httpd
$ sudo yum install createrepo
$ sudo mkdir -p /var/www/html/heylinux/RPMS/x86_64
$ cd /var/www/html/heylinux/RPMS/x86_64
$ sudo cp /home/dong.guo/rpmbuild/RPMS/x86_64/heylinux-nginx-1.1.9-el6.x86_64.rpm .
$ sudo createrepo /var/www/html/heylinux/RPMS/x86_64/
$ sudo /etc/init.d/httpd start
12. Configure the YUM source on another server
$ sudo vim heylinux.repo
[heylinux] name=heylinux baseurl=http://172.16.1.21/heylinux/RPMS/x86_64/ gpgcheck=0 enabled=1
$ sudo yum install heylinux-nginx
Loaded plugins: fastestmirror, security Loading mirror speeds from cached hostfile heylinux | 951 B 00:00 heylinux/primary | 978 B 00:00 heylinux 1/1 Setting up Install Process Resolving Dependencies --> Running transaction check ---> Package heylinux-nginx.x86_64 0:1.1.9-el6 set to be updated --> Finished Dependency Resolution Dependencies Resolved ==================================================================================================== Package Arch Version Repository Size ==================================================================================================== Installing: heylinux-nginx x86_64 1.1.9-el6 heylinux 386 k Transaction Summary ==================================================================================================== Install 1 Package(s) Upgrade 0 Package(s) Total download size: 386 k Installed size: 940 k Is this ok [y/N]: y Downloading Packages: heylinux-nginx-1.1.9-el6.x86_64.rpm | 386 kB 00:00 Running rpm_check_debug Running Transaction Test Transaction Test Succeeded Running Transaction Installing: heylinux-nginx-1.1.9-el6.x86_64 1/1 Installed: heylinux-nginx.x86_64 0:1.1.9-el6 Complete!
13. Done
AWS automation script which supports “–clone” as “Launch More Like This” on web console
Posted by mcsrainbow in Linux&Unix on December 27, 2014
I just completed an automation script for AWS, it supports “–clone” function, likes the “Launch More Like This” on AWS web console, based on Python boto library.
Address：https://github.com/mcsrainbow/python-demos/blob/master/demos/awscli.py
$ ./awscli.py -h
usage: awscli.py [-h] (--create | --clone | --terminate) --region REGION
[--instance_name INSTANCE_NAME] [--image_id IMAGE_ID]
[--instance_type INSTANCE_TYPE] [--key_name KEY_NAME]
[--security_group_ids SECURITY_GROUP_IDS]
[--subnet_id SUBNET_ID]
[--src_instance_name SRC_INSTANCE_NAME]
[--dest_instance_name DEST_INSTANCE_NAME]
[--private_ip_address PRIVATE_IP_ADDRESS]
[--instance_id INSTANCE_ID] [--volume_size VOLUME_SIZE]
[--volume_type {standard,io1,gp2}]
[--volume_zone VOLUME_ZONE] [--volume_iops VOLUME_IOPS]
[--volume_delete_on_termination]
[--load_balancer_name LOAD_BALANCER_NAME]
[--ignore_load_balancer]
[--quick]
examples:
./awscli.py --create --region us-west-1 --instance_name idc1-server2 \
--image_id ami-30f01234 --instance_type t1.micro \
--key_name idc1-keypair1 --security_group_ids sg-eaf01234f \
--subnet_id subnet-6d901234
./awscli.py --create --region us-west-1 --instance_name idc1-server3 \
--image_id ami-30f01234 --instance_type t1.micro \
--key_name idc1-keypair1 --security_group_ids sg-eaf01234f \
--subnet_id subnet-6d901234 --volume_size 10 --volume_type gp2 \
--volume_zone us-west-1a --volume_delete_on_termination \
--load_balancer_name idc1-elb1 --private_ip_address 172.16.2.23
./awscli.py --clone --region us-west-1 --src_instance_name idc1-server1 \
--dest_instance_name idc1-server2
./awscli.py --clone --region us-west-1 --src_instance_name idc1-server1 \
--dest_instance_name idc1-server3 --private_ip_address 172.16.2.23
./awscli.py --clone --region us-west-1 --src_instance_name idc1-server1 \
--dest_instance_name idc1-server3 --private_ip_address 172.16.2.23 \
--ignore_load_balancer
./awscli.py --terminate --region us-west-1 --instance_name idc1-server3
./awscli.py --terminate --region us-west-1 --instance_id i-01234abc
./awscli.py --terminate --region us-west-1 --instance_id i-01234abc --quick
...
optional arguments:
-h, --help show this help message and exit
--create create instance
--clone clone instance
--terminate terminate instance
--region REGION
--instance_name INSTANCE_NAME
--image_id IMAGE_ID
--instance_type INSTANCE_TYPE
--key_name KEY_NAME
--security_group_ids SECURITY_GROUP_IDS
--subnet_id SUBNET_ID
--src_instance_name SRC_INSTANCE_NAME
--dest_instance_name DEST_INSTANCE_NAME
--private_ip_address PRIVATE_IP_ADDRESS
--instance_id INSTANCE_ID
--volume_size VOLUME_SIZE
in GiB
--volume_type {standard,io1,gp2}
--volume_zone VOLUME_ZONE
--volume_iops VOLUME_IOPS
--volume_delete_on_termination
delete volumes on termination
--load_balancer_name LOAD_BALANCER_NAME
--ignore_load_balancer
ignore load balancer setting
--quick no wait on termination
...
optional arguments:
-h, --help show this help message and exit
--create create instance
--clone clone instance
--terminate terminate instance
--region REGION
--instance_name INSTANCE_NAME
--image_id IMAGE_ID
--instance_type INSTANCE_TYPE
--key_name KEY_NAME
--security_group_ids SECURITY_GROUP_IDS
--subnet_id SUBNET_ID
--src_instance_name SRC_INSTANCE_NAME
--dest_instance_name DEST_INSTANCE_NAME
--private_ip_address PRIVATE_IP_ADDRESS
--instance_id INSTANCE_ID
--volume_size VOLUME_SIZE
in GiB
--volume_type {standard,io1,gp2}
--volume_zone VOLUME_ZONE
--volume_iops VOLUME_IOPS
--volume_delete_on_termination
delete volumes on termination
--load_balancer_name LOAD_BALANCER_NAME
--quick no wait on termination
$ ./awscli.py –create –region us-west-1 –instance_name idc1-server1 –image_id ami-30f01234 \
–instance_type t1.micro –key_name idc1-keypair1 –security_group_ids sg-eaf01234f \
–subnet_id subnet-6d901234 –volume_size 10 –volume_type gp2 –volume_zone us-west-1a \
–volume_delete_on_termination –load_balancer_name idc1-elb1 –private_ip_address 172.16.2.21
1. Launching instance: idc1-server1
2. Creating tag as instance name: {"Name": idc1-server1}
Instance state: pending
Instance state: running
3. Creating secondary volume for instance: idc1-server1 as gp2 10G
Volume status: available
4. Attaching volume: vol-4ba6a54c to instance: idc1-server1 as device: /dev/sdf
5. Adding instance: idc1-server1 to ELB: idc1-elb1
$ ./awscli.py –clone –region us-west-1 –src_instance_name idc1-server1 –dest_instance_name idc1-server2
1. Launching instance: idc1-server2
2. Creating tag as instance name: {"Name": idc1-server2}
Instance state: pending
Instance state: running
3. Creating secondary volume for instance: idc1-server2 as gp2 10G
Volume status: available
4. Attaching volume: vol-5b61635c to instance: idc1-server2 as device: /dev/sdf
5. Adding instance: idc1-server2 to ELB: idc1-elb1
$ ./awscli.py –terminate –region us-west-1 –instance_name idc1-server2
Terminating instance: idc1-server2 id: i-86976d62 Instance state: shutting-down Instance state: shutting-down Instance state: terminated
Create an Ansible Module and then use Module Provided ‘Facts’
Posted by mcsrainbow in Linux&Unix on July 30, 2014
Sometimes the default Ansible Facts are not enough.
For example, Ansible doesn’t provide “ansible_private_ipv4_address” which with private IP address.
With it, we don’t need to worry about which IP on which NIC should be used.
But we can create an Ansible module and then use module provided ‘Facts’
The steps:
[root@idc-server2 ~]# ifconfig
eth0 Link encap:Ethernet HWaddr 1B:2B:3B:4B:5B:6B
inet addr:172.16.1.2 Bcast:172.16.1.255 Mask:255.255.252.0
...
eth1 Link encap:Ethernet HWaddr 1A:2A:3A:4A:5A:6A
inet addr:100.100.100.100 Bcast:100.100.100.255 Mask:255.255.255.240
...
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
...
[root@idc-server1 ansible]# vim myfacts.yml
--- - hosts: idc-server2 roles: - myfacts
[root@idc-server1 ansible]# mkdir -p roles/myfacts/{tasks,templates}
[root@idc-server1 ansible]# vim roles/myfacts/tasks/main.yml
--- - name: run myfacts module to get customized facts myfacts: get_facts=yes - name: update file with the customized facts template: src=myfacts.txt.j2 dest=/tmp/myfacts.txt
[root@idc-server1 ansible]# vim roles/myfacts/templates/myfacts.txt.j2
ansible_private_ipv4_address : {{ ansible_private_ipv4_address }}
[root@idc-server1 ansible]# mkdir -p library/heylinux
[root@idc-server1 ansible]# vim library/heylinux/myfacts
#!/usr/bin/python
import json
import commands
import re
def get_ansible_private_ipv4_address():
iprex = "(^192\.168)|(^10\.)|(^172\.1[6-9])|(^172\.2[0-9])|(^172\.3[0-1])"
output = commands.getoutput("""/sbin/ifconfig |grep "Link encap" |awk '{print $1}' |grep -wv 'lo'""")
nics = output.split('\n')
for i in nics:
ipaddr = commands.getoutput("""/sbin/ifconfig %s |grep -w "inet addr" |cut -d: -f2 | awk '{print $1}'""" % (i))
if re.match(iprex,ipaddr):
ansible_private_ipv4_address = ipaddr
return ansible_private_ipv4_address
def main():
global module
module = AnsibleModule(
argument_spec = dict(
get_facts=dict(default="yes", required=False),
),
supports_check_mode = True,
)
ansible_facts_dict = {
"changed" : False,
"ansible_facts": {
}
}
if module.params['get_facts'] == 'yes':
ansible_private_ipv4_address = get_ansible_private_ipv4_address()
ansible_facts_dict['ansible_facts']['ansible_private_ipv4_address'] = ansible_private_ipv4_address
print json.dumps(ansible_facts_dict)
from ansible.module_utils.basic import *
from ansible.module_utils.facts import *
main()
[root@idc-server1 ansible]# ansible-playbook -u root myfacts.yml -i hosts
PLAY [idc1-server2] *************************************************************** GATHERING FACTS *************************************************************** ok: [idc1-server2] TASK: [myfacts | run myfacts module to get customized facts] ************** ok: [idc1-server2] TASK: [myfacts | update file with the customized facts] ********************* changed: [idc1-server2] PLAY RECAP ******************************************************************** idc1-server2 : ok=3 changed=1 unreachable=0 failed=0
[root@idc-server1 ansible]# ssh idc1-server2 ‘cat /tmp/myfacts.txt’
ansible_private_ipv4_address : 172.16.1.2
Setup GlusterFS with Distributed Replicated Volumes and Native client
Posted by mcsrainbow in Linux&Unix on July 28, 2014
OS: CentOS 6.4 x86_64 Minimal
1. Install packages, on idc1-server{1-4}:
# wget -P /etc/yum.repos.d http://download.gluster.org/pub/gluster/glusterfs/LATEST/CentOS/glusterfs-epel.repo
# yum install -y glusterfs glusterfs-server glusterfs-fuse
# /etc/init.d/glusterd start
# chkconfig glusterd on
2. Configure peers, just on idc1-server1:
[root@idc1-server1 ~]# gluster peer probe idc1-server2
peer probe: success
[root@idc1-server1 ~]# gluster peer probe idc1-server3
peer probe: success
[root@idc1-server1 ~]# gluster peer probe idc1-server4
peer probe: success
NOTE:
The following step is very important with the version 3.4.3!
If configured all peers with hostname on one host, must remember to recreate the peer of this host on another host.
Otherwise all other hosts cannot recognize this host by hostname.
10.1.1.35 is the ip address of idc1-server1, detach it then probe with hostname.
Just on idc1-server2:
[root@idc1-server2 ~]# gluster peer detach 10.1.1.35
peer detach: success
[root@idc1-server2 ~]# gluster peer probe idc1-server1
peer probe: success
Just on idc1-server2:
[root@idc1-server2 ~]# gluster peer status
Number of Peers: 3 Hostname: idc1-server3 Uuid: 01f25251-9ee6-40c7-a322-af53a034aa5a State: Peer in Cluster (Connected) Hostname: idc1-server4 Uuid: 212295a6-1f38-4a1e-968c-577241318ff1 State: Peer in Cluster (Connected) Hostname: idc1-server1 Port: 24007 Uuid: ed016c4e-7159-433f-88a5-5c3ebd8e36c9 State: Peer in Cluster (Connected)
3. Create directories, on idc1-server{1-4}:
# mkdir -p /usr/local/share/{datavolume1,datavolume2,datavolume3}
# chown -R root:root /mnt/{datavolume1,datavolume2,datavolume3}
# ls -l /usr/local/share/
total 24 drwxr-xr-x. 2 root root 4096 Sep 23 2011 applications drwxr-xr-x 2 root root 4096 Apr 1 12:19 datavolume2 drwxr-xr-x. 2 root root 4096 Sep 23 2011 info drwxr-xr-x. 21 root root 4096 Feb 20 2013 man drwxr-xr-x 2 root root 4096 Apr 1 12:19 datavolume1 drwxr-xr-x 2 root root 4096 Apr 1 12:19 datavolume3
4. Create Distributed Replicated Volumes, just on idc1-server1:
[root@idc1-server1 ~]# gluster volume create datavolume1 replica 2 transport tcp idc1-server1:/usr/local/share/datavolume1 idc1-server2:/usr/local/share/datavolume1 idc1-server3:/usr/local/share/datavolume1 idc1-server4:/usr/local/share/datavolume1 force
volume create: datavolume1: success: please start the volume to access data
[root@idc1-server1 ~]# gluster volume create datavolume2 replica 2 transport tcp idc1-server1:/usr/local/share/datavolume2 idc1-server2:/usr/local/share/datavolume2 idc1-server3:/usr/local/share/datavolume2 idc1-server4:/usr/local/share/datavolume2 force
volume create: datavolume2: success: please start the volume to access data
[root@idc1-server1 ~]# gluster volume create datavolume3 replica 2 transport tcp idc1-server1:/usr/local/share/datavolume3 idc1-server2:/usr/local/share/datavolume3 idc1-server3:/usr/local/share/datavolume3 idc1-server4:/usr/local/share/datavolume3 force
volume create: datavolume3: success: please start the volume to access data
[root@idc1-server1 ~]# gluster volume start datavolume1
volume start: datavolume1: success
[root@idc1-server1 ~]# gluster volume start datavolume2
volume start: datavolume2: success
[root@idc1-server1 ~]# gluster volume start datavolume3
volume start: datavolume3: success
[root@idc1-server1 ~]# gluster volume info
Volume Name: datavolume1 Type: Distributed-Replicate Volume ID: aea76c2a-b754-4037-9634-c2062e2955c3 Status: Started Number of Bricks: 2 x 2 = 4 Transport-type: tcp Bricks: Brick1: idc1-server1:/usr/local/share/datavolume1 Brick2: idc1-server2:/usr/local/share/datavolume1 Brick3: idc1-server3:/usr/local/share/datavolume1 Brick4: idc1-server4:/usr/local/share/datavolume1 Volume Name: datavolume2 Type: Distributed-Replicate Volume ID: 1ed65c6e-ee23-475a-82c7-2967e2fc2569 Status: Started Number of Bricks: 2 x 2 = 4 Transport-type: tcp Bricks: Brick1: idc1-server1:/usr/local/share/datavolume2 Brick2: idc1-server2:/usr/local/share/datavolume2 Brick3: idc1-server3:/usr/local/share/datavolume2 Brick4: idc1-server4:/usr/local/share/datavolume2 Volume Name: datavolume3 Type: Distributed-Replicate Volume ID: b63bb4ea-bd37-4dd6-9a4c-230e6d236afa Status: Started Number of Bricks: 2 x 2 = 4 Transport-type: tcp Bricks: Brick1: idc1-server1:/usr/local/share/datavolume3 Brick2: idc1-server2:/usr/local/share/datavolume3 Brick3: idc1-server3:/usr/local/share/datavolume3 Brick4: idc1-server4:/usr/local/share/datavolume3
5. Configure the Network ACL, just on idc1-server1:
[root@idc1-server1 ~]# gluster volume set datavolume1 auth.allow 10.1.1.*,10.1.2.*
6. Tests on idc1-client15, install the version ‘-3.4.0.57rhs-1.el6_5’ which supports option: ‘backup-volfile-servers’
[root@idc1-client15 ~]# yum install -y glusterfs-3.4.0.57rhs-1.el6_5 glusterfs-libs-3.4.0.57rhs-1.el6_5 glusterfs-fuse-3.4.0.57rhs-1.el6_5
[root@idc1-client15 ~]# mkdir -p /mnt/{datavolume1,datavolume2,datavolume3}
[root@idc1-client15 ~]# chown -R root:root /mnt/{datavolume1,datavolume2,datavolume3}
[root@idc1-client15 ~]# mount -t glusterfs -o backup-volfile-servers=idc1-server2:idc1-server3:idc1-server4,ro idc1-server1:datavolume1 /mnt/datavolume1/
[root@idc1-client15 ~]# mount -t glusterfs -o backup-volfile-servers=idc1-server2:idc1-server3:idc1-server4,ro idc1-server1:datavolume2 /mnt/datavolume2/
[root@idc1-client15 ~]# mount -t glusterfs -o backup-volfile-servers=idc1-server2:idc1-server3:idc1-server4,ro idc1-server1:datavolume3 /mnt/datavolume3/
[root@idc1-client15 ~]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/vg_t-lv_root
59G 7.7G 48G 14% /
tmpfs 3.9G 0 3.9G 0% /dev/shm
/dev/xvda1 485M 33M 428M 8% /boot
idc1-server3:datavolume1 98G 8.6G 85G 10% /mnt/datavolume1
idc1-server3:datavolume2 98G 8.6G 85G 10% /mnt/datavolume2
idc1-server3:datavolume3 98G 8.6G 85G 10% /mnt/datavolume3
Tests:
Write on idc1-client15 mount point /mnt/datavolume1:
[root@idc1-client15 ~]# umount /mnt/datavolume1
[root@idc1-client15 ~]# mount -t glusterfs idc1-server3:datavolume1 /mnt/datavolume1/
[root@idc1-client15 ~]# echo “This is idc1-client15” > /mnt/datavolume1/hello.txt
[root@idc1-client15 ~]# mkdir /mnt/testdir
[root@idc1-server1 ~]# ls /usr/local/share/datavolume1/
hello.txt testdir
Saw hello.txt and testdir
Write on idc1-server1 volumes dir /usr/local/share/datavolume1:
[root@idc1-server1 ~]# echo “This is idc1-server1” > /usr/local/share/datavolume1/hello.2.txt
[root@idc1-server1 ~]# mkdir /usr/local/share/datavolume1/test2
[idc1-client15][root@idc1-client15 ~]# ls /mnt/datavolume1
[idc1-client15][root@idc1-client15 datavolume1]# ls -l /mnt/datavolume1
hello.txt testdir
Didn’t see hello.2.txt and test2
Write on idc1-server1 mount point /mnt/datavolume1:
[root@idc1-server1 ~]# mount -t glusterfs idc1-server1:datavolume1 /mnt/datavolume1/
[root@idc1-server1 ~]# echo “This is idc1-server1” > /mnt/datavolume1/hello.3.txt
[root@idc1-server1 ~]# mkdir /mnt/datavolume1/test3
[idc1-client15][root@idc1-client15 datavolume1]# ls /mnt/datavolume1
hello.2.txt hello.3.txt hello.txt test2 test3 testdir
Saw hello.3.txt and test3, and the hello.2.txt and test2 also.
So I guess if we just write or delete file outside the mount point, it doesn’t notify other nodes, so the changes didn’t take effect.
We should mount it on all servers which we want to write or delete files.
Other Notes:
Delete:
# gluster volume stop datavolume1
# gluster volume delete datavolume1
Move:
# gluster peer detach idc1-server4
ACL:
# gluster volume set datavolume1 auth.allow 10.1.1.*,10.1.2.*
Add:
# gluster peer probe idc1-server5
# gluster peer probe idc1-server6
# gluster volume add-brick datavolume1 idc1-server5:/usr/local/share/datavolume1 idc1-server6:/usr/local/share/datavolume1
Shrink and migration:
# gluster volume remove-brick datavolume1 idc1-server1:/usr/local/share/datavolume1 idc1-server5:/usr/local/share/datavolume1 start
# gluster volume remove-brick datavolume1 idc1-server1:/usr/local/share/datavolume1 idc1-server5:/usr/local/share/datavolume1 status
# gluster volume remove-brick datavolume1 idc1-server1:/usr/local/share/datavolume1 idc1-server5:/usr/local/share/datavolume1 commit
Rebalance:
# gluster volume rebalance datavolume1 start
# gluster volume rebalance datavolume1 status
# gluster volume rebalance datavolume1 stop
If idc1-server1 dead:
# gluster volume replace-brick datavolume1 idc1-server1:/usr/local/share/datavolume1 idc1-server5:/usr/local/share/datavolume1 commit -force
# gluster volume heal datavolume1 full
Install Zimbra MTA 8 on CentOS 6.5
Posted by mcsrainbow in Corporation, Linux&Unix on May 5, 2014
Just a copy from http://scrble.net/install-zimbra-mta-8-on-centos-6-5/
Make sure no mail server and http server is running. Otherwise stop them and disable or uninstall them.
Make sure SELINUX is disabled. By default it is disabled on CentOS 6.5. To confirm it, check using getforce
# getenforce
Disabled
# cat /etc/sysconfig/selinux
# This file controls the state of SELinux on the system. # SELINUX= can take one of these three values: # enforcing - SELinux security policy is enforced. # permissive - SELinux prints warnings instead of enforcing. # disabled - SELinux is fully disabled. SELINUX=disabled # SELINUXTYPE= type of policy in use. Possible values are: # targeted - Only targeted network daemons are protected. # strict - Full SELinux protection. SELINUXTYPE=targeted
Stop and Disable Postfix and Iptables
# service postfix stop
# service ip6tables stop
# service iptables stop
# chkconfig postfix off
# chkconfig ip6tables off
# chkconfig iptables off
Make sure that domain name is set in correct format at /etc/hosts file. The format is:
ip_address server.domain.com server
Also set the hostname at /etc/sysconfig/network and also:
# service network restart
# hostname server.domain.com
Make sure you have proper MX record configured for your domain
# host -t mx server.domain.com
server.domain.com mail is handled by 10 server.domain.com.
Install the required packages:
# yum install perl nc sysstat -y
Download the latest Zimbra MTA and extract it:
# cd /opt/
# wget http://files2.zimbra.com/downloads/8.0.6_GA/zcs-8.0.6_GA_5922.RHEL6_64.20131203103705.tgz
# tar -zxvf zcs-8.0.6_GA_5922.RHEL6_64.20131203103705.tgz
# cd zcs-8.0.6_GA_5922.RHEL6_64.20131203103705
Start the installation:
# ./install.sh –platform-override
Just accept all default: Do you agree with the terms of the software license agreement? [N] Y Do you agree with the terms of the software license agreement? [N] Y Select the packages to install Install zimbra-ldap [Y] Install zimbra-logger [Y] Install zimbra-mta [Y] Install zimbra-snmp [Y] Install zimbra-store [Y] Install zimbra-apache [Y] Install zimbra-spell [Y] Install zimbra-memcached [N] Install zimbra-proxy [N]
Give “Y” when it asked the following:
The system will be modified. Continue? [N] Y
Set the Admin password:
Password for admin@server.domain.com (min 6 characters): [_yTalZUL9OZ] password
Finally press ‘a’ to complete the installation.
*** CONFIGURATION COMPLETE - press 'a' to apply Select from menu, or press 'a' to apply config (? - help) a Save configuration data to a file? [Yes] Save config in file: [/opt/zimbra/config.26430] Saving config in /opt/zimbra/config.26430...done. The system will be modified - continue? [No] YES Configuration complete - press return to exit
Switch user as zimbra
# su – zimbra
Start Zimbra services:
$ zmcontrol start
Host server.domain.com
Starting zmconfigd...Done.
Starting logger...Done.
Starting mailbox...Done.
Starting antispam...Done.
Starting antivirus...Done.
Starting opendkim...Done.
Starting snmp...Done.
Starting spell...Done.
Starting mta...Done.
Starting stats...Done.
Check Zimbra services status:
$ zmcontrol status
Host server.domain.com
antispam Running
antivirus Running
ldap Running
logger Running
mailbox Running
mta Running
opendkim Running
snmp Running
spell Running
stats Running
zmconfigd Running
You can check and manage through Zimbra Admin Control Panel at https://server.domain.com:7071 and client web-mail at https://server.domain.com
Recent Comments
pls tell me how to lock the target directory of the bbcp user?
Hi, i am trying to send out mail to office 365 and i am getting below error…
How to do the above settings for windows 7 machine and jenkins hosted on tomcat installed on the same machine??
Hey pal, who said pfSense sucks screw you!
Hey dear can we done some Project together ? i am from iran and i have some…
Upload it to Mega.nz
How can I replicate this issue.? I also faced the same problem, where all the cron entries lost. Not sure…
Hi, I am trying to integrate Jenkins with office365 and as mentioned above, (please correct me if i am…
Hi, I am trying to integrate Jenkins with office365 and as mentioned above, (please correct me if i am…
Vishal : shreyas :…