Skip to content

2014

Puppet Master-Agent Installation on FreeBSD

Puppet is a software which can automate configuration and management of machines and software running on them. This tool has great benefits for system administrator because it helps sysadmin to be the dream of every sysadmin, a lazy sysadmin. Puppet has great support for many operating system. Unfortunately its installation on my favourite OS, FreeBSD, is not so smooth. An introduction of Puppet installation which I found in BSD Magazine January 2012 edition is a starting point but I have to make some modification due to some of deprecated configurations. So, here I want to show you how to install and configure the basic of Puppet in FreeBSD in its master-agent scenario.

Let's start...

Puppet benefits:

  1. automated server installation
  2. mass deployment of changes to servers
  3. maintain server state consistency

Puppet scenario: 0. FreeBSD (master)

  1. FreeBSD (agent)

FreeBSD initial:

update ports

# portsnap fetch extract
# portsnap fetch update

install portmaster via ports

# cd /usr/ports/ports-mgmt/portmaster
# make install clean
# rehash

switch to pkgng

# portmaster -dB ports-mgmt/pkg

convert package database to new pkgng format

# pkg2ng

use pkgng by default

# echo 'WITH_PKGNG=yes' >> /etc/make.conf

define new repository for pkgng

# mkdir -p /usr/local/etc/pkg/repos
# cat << 'EOF' > /usr/local/etc/pkg/repos/FreeBSD.conf
 FreeBSD: {
     url: "http://pkg.FreeBSD.orf/${ABI}/latest",
        mirror_type: "srv",
     enabled: true
    }
 EOF
# pkg update

test pkgng

# pkg install sl
# sl

Puppet master initial:

IMPORTANT: Puppet requires FQDN hostname install from ports

# cd /usr/ports/sysutils/puppet
# make install clean

generate puppet configuration file

# puppet master --genconfig > /usr/local/etc/puppet/puppet.conf

copy auth.conf-dist to auth.conf

# cp /usr/local/etc/puppet/auth.conf-dist /usr/local/etc/puppet/auth.conf

some "red" warnings when run puppet help

Warning: Setting manifestdir is deprecated. See http://links.puppetlabs.com/env-settings-deprecations
   (at /usr/local/lib/ruby/site_ruby/1.9/puppet/settings.rb:1095:in `block in issue_deprecations')
Warning: Setting manifest is deprecated in puppet.conf. See http://links.puppetlabs.com/env-settings-deprecations
   (at /usr/local/lib/ruby/site_ruby/1.9/puppet/settings.rb:1095:in `block in issue_deprecations')
Warning: Setting modulepath is deprecated in puppet.conf. See http://links.puppetlabs.com/env-settings-deprecations
   (at /usr/local/lib/ruby/site_ruby/1.9/puppet/settings.rb:1095:in `block in issue_deprecations')
Warning: Setting templatedir is deprecated. See http://links.puppetlabs.com/env-settings-deprecations
   (at /usr/local/lib/ruby/site_ruby/1.9/puppet/settings.rb:1095:in `block in issue_deprecations')

skeleton files creation

# mkdir -p /usr/local/etc/puppet/files
# mkdir -p /usr/local/etc/puppet/manifests
# touch /usr/local/etc/puppet/fileserver.conf
# touch /usr/local/etc/puppet/files/sudoers
# touch /usr/local/etc/puppet/manifests/site.pp
# touch /usr/local/etc/puppet/environments/production/environment.conf
# touch /usr/local/etc/puppet/environments/production/manifests/site.pp

edit puppet.conf

[main]
    environmentpath = /usr/local/etc/puppet/environments
[master]
    #manifestdir = /usr/local/etc/puppet/manifests
    #manifest = /usr/local/etc/puppet/manifests/site.pp
    #modulepath = /usr/local/etc/puppet/modules:/usr/share/puppet/modules
    #templatedir = /var/puppet/templates
    pluginsource = puppet://puppetmaster.domain/plugins
    pluginfactsource = puppet://puppetmaster.domain/pluginfacts

edit fileserver.conf

[files]
    path /usr/local/etc/puppet/files
    allow *.domain

edit environments/production/environment.conf

modulepath = /usr/local/etc/puppet/modules:/usr/share/puppet/modules
manifest = /usr/local/etc/puppet/environments/production/manifests/site.pp
### EXPERIMENTAL ###
#config_version = get_environment_commit.sh
#environment_timeout = 5s

edit environments/production/manifests/site.pp

class sudoers {
        file { "/usr/local/etc/sudoers":
                ensure  => file,
                owner   => root,
                group   => wheel,
                mode    => 440,
                source  => "puppet://puppetmaster.domain/files/sudoers",
        }
}

node 'puppetclient.domain' {
        include sudoers
}

start puppet master

# echo 'puppetmaster_enable="YES"' >> /etc/rc.conf
# /usr/local/etc/rc.d/puppetmaster start

sign the certificate when agent initiate a certificate signing session # puppet cert --list --all # puppet cert --sign puppetagent.domain

edit files/sudoers

root ALL=(ALL) ALL

update puppet agents using kick

# puppet kick puppetagent.domain

Puppet agent initial:

IMPORTANT: Puppet requires FQDN hostname install from ports # cd /usr/ports/sysutils/puppet # make install clean

configure hostname

# hostname puppetagent.domain

create and edit new auth.conf

path /run
method save
allow puppetmaster.domain

initiate a certificate signing session from agent to master and wait to be signed and automatically closed

# puppet agent -v --server puppetmaster.domain --waitforcert 60 --test

enable puppet agent on rc.conf

# echo 'puppet_enable="YES"' >> /etc/rc.conf
# echo 'puppet_flags="-v --listen --server puppetmaster.domain"' >> /etc/rc.conf

start puppet agent

# /usr/local/etc/rc.d/puppet start

inspect /usr/local/etc/sudoers after puppet master kicked some changes

# less /usr/local/etc/sudoers

Puppet problems:

cannot connect to https://forge.puppetlabs.com

security/ca_root_nss port needed to be installed with ETCSYMLINK turned on

puppet kick and --listen flag is deprecated still can't find good configuration for using sysutils/mcollective-puppet-agent as alternative


References: http://www.iceflatline.com/2013/02/how-to-use-portmaster-to-update-ports/ http://www.fitzdsl.net/2013/11/utiliser-pkgng-sous-freebsd-avec-puppet/ https://mebsd.com/make-build-your-freebsd-word/pkgng-first-look-at-freebsds-new-package-manager.html https://forums.freebsd.org/viewtopic.php?&t=36732 http://www.6tech.org/2013/01/how-to-install-puppet-open-source-on-centos-6-3/ https://github.com/puppetlabs-operations/puppet-freebsd

DRBD: Extend DRBD Disk Online

When using DRBD, we can grow DRBD disk online so we do not need to disturb the production process we have in the server. The requirement to this feature is the backing block device can be resized online so it is possible to resize the DRBD disk. There are two criterias that must be filled.

1. The backing device must be managed by a logical volume manager such as LVM
2. The resource must currently be in the Connected connection state.

First, we need to grow the backing device on both nodes and make sure only one node in Primary node.

# lvextend -L 16GB /dev/VolGroup00/LogVol01

Then, resize the DRBD service to trigger a synchronization of the new section from primary node to secondary node.

# drbdadm resize <resource>

If the additional space is clean, we can skip syncing the additional space by using the --assume-clean option.

# drbdadm -- --assume-clean resize <resource>

After that resize the file system of DRBD device.

# resize2fs /dev/drbdX

IMPORTANT: Always create backup, better safe than sorry.

References: http://www.drbd.org/users-guide/s-resizing.html http://lists.linbit.com/pipermail/drbd-user/2008-August/009908.html

Git Tutorial

Git is a distributed version control system version control system track history of a collection of files and includes the functionality to revert to another version.

Distributed version control system does not necessarily have a central server which stores data.

User can copy existing repository (cloning).

Every clone contains full history of the collection of files and a clone repository has the same functionality as the original repository.

Users with sufficient authorization can push changes from their local repositories to remote repositories, they can also fetch or pull changes from other repositories to their local Git repository.

Git support branching which means you can work on different versions of your collection files.

Working tree is the current collection of files.

Git using SHA-1 checksum as file integrity protection.

Add changes to your Git repository by first add selected file to staging area and commit the changes in staging area to Git repository.

Mark changes in the working tree by staging or add changes to staging area.

# git add .

After adding files to staging area you can commit this file to permanently add them to Git repository.

# git commit

Push the modified version of files to remote repository using push command.

# git push <repository>

Heartbeat Using Different UDP Ports for Different Clusters on Same Network

Heartbeat is a daemon that provides cluster infrastructure (communication and membership) services to its clients. This allow clients to know appearance or disappearance of peer processes on other machines and to easily exchange message with them.(Heartbeat - Linux-HA)

Heartbeat clusters work only one on same network so if we setup different cluster (different authkeys) on same network it will show up as an error in log file. We can still setup more than one cluster in same network by setting different communication port for heartbeat.

The following example show configuration file for two different clusters in same network.

host configuration

192.168.123.1 host1.cluster1 192.168.123.2 host2.cluster1 192.168.123.3 host1.cluster2 192.168.123.4 host2.cluster2

first cluster
/etc/ha.d/ha.cf

debugfile /var/log/ha-debug logfile /var/log/ha-log logfacility local0 keepalive 2 deadtime 20 udpport 694 udp eth0 bcast eth0 node host1.cluster1 host2.cluster1 auto_failback on debug 1

second cluster
/etc/ha.d/ha.cf

debugfile /var/log/ha-debug logfile /var/log/ha-log logfacility local0 keepalive 2 deadtime 20 udpport 695 udp eth0 bcast eth0 node host1.cluster2 host2.cluster2 auto_failback on debug 1 Reference: http://www.gossamer-threads.com/lists/linuxha/users/57074

Installing Apache HTTP Server on FreeBSD

Installation using FreeBSD ports

Login as root, then to make sure our server's hostname can be identified locally we need to edit /etc/hosts.

# ee /etc/hosts
::1               localhost localhost.example.com
127.0.0.1         localhost localhost.example.com
192.168.1.11      host.example.com

Install Apache HTTP Server using following command, choose default for options, select OK.

# cd /usr/ports/www/apache22
# make config; make install clean
# rehash

After installation proses finished, change Apache configuration file.

# ee /usr/local/etc/apache22/httpd.conf
ServerAdmin you@example.com
ServerName host.example.com:80

To enable SSL support, uncomment following line.

Include etc/apache22/extra/httpd-ssl.conf

Save, then exit ee and open Apache' SSL configuration file.

# ee /usr/local/etc/apache22/extra/httpd-ssl.conf
ServerName host.example.com:443
ServerAdmin you@example.com
SSLCertificateFile /usr/local/openssl/certs/host.example.com-cert.pem

SSLCertificateKeyFile /usr/local/openssl/certs/host.example.com-unencrypted-key.pem

Save and exit.

Testing Apache HTTP Server Installation

Check possible error on configuration files.

# apachectl configtest

If it shows Syntax OK then there is no error in configuration files. Change /etc/rc.conf so that Apache can start at boot time. # ee /etc/rc.conf apache22_enable="YES" apache22_http_accept_enable="YES"

Save and exit from text editor and start Apache with following command.

# /usr/local/etc/rc.d/apache22 start

Test Apache by opening our domain in internet browser.

http://host.example.com/

Test SSL support by opening our domain in HTTPS protocol. https://host.example.com/

Installing Cyrus SASL Server on FreeBSD

Instalation using FreeBSD ports

Login as root then enter ports directory of Cyrus SASL and run following command.

# cd /usr/ports/security/cyrus-sasl2-saslauthd
# make config ; make install clean
# rehash

Create file smtpd.conf in directory /usr/local/lib/sasl2/. # ee /usr/local/lib/sasl2/smtpd.conf

Then add following lines.

pwcheck_method: saslauthd
mech_list: plain login

Edit file rc.conf so that SASL Authentification server can start at boot time.

# ee /etc/rc.conf
saslauthd_enable="YES"
saslauthd_flags="-a pam"

if you want to use other authentication mechanism such as LDAP, use following flags.

saslauthd_flags="-a ldap"

Save and run the startup script.

# /usr/local/etc/rc.d/saslauthd start
# /usr/local/etc/rc.d/saslauthd status

If SASL Authentification server is running, the terminal will show output similar to this.

saslauthd is running as pid 1234.

Installing MySQL Server on FreeBSD

Installation using FreeBSD ports

Login as root, then to make sure our server's hostname can be identified locally we need to edit /etc/hosts.

# ee /etc/hosts
::1               localhost localhost.example.com
127.0.0.1         localhost localhost.example.com
192.168.1.11      host.example.com

Install MySQL Server with following command.

# cd /usr/ports/databases/mysql56-server
# make -D BUILD_OPTIMIZED install clean
# rehash

After installation finished, run script for installing main database and tables used by MySQL.

# mysql_install_db --user=mysql –basedir=/usr/local/

Run following command to start MySQL daemon and create password for MySQL's root. # mysqld_safe & # mysqladmin -u root password 'localpassword' # mysqladmin -u root -h host.example.com password 'remotepassword'

MySQL has three types of configuration files i.e. my-small.cnf, my-medium.cnf, my-large.cnf, and my-huge.cnf. Copy suitable configuration file to /var/db/mysql.

# cp /usr/local/share/mysql/my-medium.cnf /var/db/mysql/my.cnf

To deactivate MySQL TCP Networking edit MySQL configuration file.

# ee /var/db/mysql/my.cnf
skip-networking

Save and exit text editor.

Test MySQL Server Installation

In order MySQL to start at boot time, edit file /etc/rc.conf

# ee /etc/rc.conf
mysql_enable="YES"

Restart MySQL to apply change in configuration file.

# /usr/local/etc/rc.d/mysql-server restart

For checking if MySQL is running correctly, run following command.

# mysqlshow –p

If MySQL running then the following output will appear.

+---------------------------+
|         Databases         |
+---------------------------+
| information_schema        |
| mysql                     |
| test                      |
+---------------------------+

If there is error then the error log is saved in file /var/db/mysql/host.example.com.err. Check the permission of /tmp directory.

# ls -ld /tmp
drwxrwxrwt    7 root  wheel      512 Feb 17 12:00 /tmp

If the output is not the same with line above, repair it with following command.

# chown root:wheel /tmp
# chmod 777 /tmp
# chmod =t /tmp

Installation of MySQL Server is finished.

DRBD: Manual Split Brain Recovery

Split Brain in DRBD is a condition where each host think that resource in another host is outdated, thus synchronization won't be occured. This condition often occurs because of network or power failure which is indicated by these variables:

log: Split-Brain detected, dropping connection
connection-state: StandAlone/WFConnection

To manually recover the cluster from split brain condition, the following steps is required.

1. on victim connection state must be StandAlone
# drbdadm disconnect <resource>
2. set victim as secondary
# drbdadm secondary <resource>
3. reconnect victim
# drbdadm connect --discard-my-data <resource>
4. on the survivor, if the connection state StandAlone enter following command
# drbdadm connect <resource>

Reference: http://www.drbd.org/users-guide-8.4/s-resolve-split-brain.html

DRBD: Troubleshooting and Error Recovery of Hard Drive Failure or Replacement

DRBD refers to block devices designed as a building block to form high availability cluster. This is done by mirroring a whole block device via an assigned network. DRBD can be understood as network based RAID-1. (DRBD)

Sometimes we have a disk failure on hard drive which contains DRBD backing device. The following steps can be used to replace or recover the failed drive.

1. detach drbd resource from broken backing storage
# drbdadm detach <resource>
2. check state of drbd disk
# drbdadm dstate <resource>
Diskless/UpToDate
3. if using internal meta data, bind DRBD device to new hard disk
# drbdadm create-md <resource>
# drbdadm attach <resource>
4. if using external meta data DRBD unable to recognize hard drive was swapped, need additional step
# drbdadm create-md <resource>
# drbdadm attach <resource>
# drbdadm invalidate <resource>

Reference: http://www.drbd.org/users-guide-8.4/ch-troubleshooting.html

FreeBSD Check TCP/UDP Open Port or Services

In this short tutorial I will show how to check TCP/UDP open port or services in FreeBSD box. For checking open port or services which uses port in FreeBSD simply issue the following commands.

  1. check TCP connection
sockstat -Ptcp
  1. check UDP connection
sockstat -Pudp

Combine with grep to search port used by specific service or service which using specific port.

  1. use grep to get service or port
sockstat -Ptcp | grep <port number>
sockstat -Ptcp | grep -i <service name>
sockstat -Pudp | grep <port number>
sockstat -Pudp | grep -i <service name>