Chapter 1
Getting Started with Puppet
1.1 How Puppet Works
-Ravi is a DevOps person in charge of managing company
infrastructure
- He manually configure or do changes on all the virtual
machines
- It takes weeks or may be months depending on the size of
infrastructure.
-Is there a
way to manage this better?
- What if he
can deploy change concurrently to all the systems
Puppet helps
to automate repetitive and error prone tasks
-Provision
servers
-Deploy application
-Proactively
manage infrastructure
It helps to
automate full infrastructure- Like OS, network, middleware, application.
Ravi can
manage resources by writing in Puppet’s Simple Language. It will work on Linux,
Solaris, AIX, windows,etc.
It also
possible to deploy in different env, like public, private and hybrid clouds.
1.2 What is Puppet?
·
Ruby based
·
Can run in Client-Server or standalone model
·
Developed by Luke Kanies
·
used to manage a host throughout its lifecycle:
from initial build and installation, to upgrades, maintenance, and finally to
end-of-life
·
designed to continuously interact with your
hosts, unlike provisioning tools which build your hosts and leave them
unmanaged.
·
Uses puppet declarative language( focus is what
instead of how)
Four Major challenges in keeping up the configuration of
systems and application:
1>
ensuring that servers we deploy have consistent base
configurations
2>
remediating inconsistent or incorrect
configurations
3>
deploying applications in a consistent and
reliable way
4>
automating all of these things
Three Components of puppet operating model:
1>
Deployment
2>
Configuration Language and resource Abstraction
Layer
3>
Transactional Layer
1.3 Why can’t we achieve above things by script
·
Take lot of time – Write, Test and Debug
·
Procedural – goes in multiple of line to achieve
a result, focus on how.
·
Mulitple ways of doing same thing, difficult for
other person to understand
·
Need different code for different OS
·
For Same OS, different version have diff
commands
1.4 Why Puppet?
·
Overcomes shortcomings of other configuration
management tools.
e.g: Chef- Ruby is req, CEconfig- Complex
·
Puppet programs are Declarative
·
Focus is “What” instead of “How”, less complex
1.5 Change Management
·
Enforcing
a particular state
·
To
maintain consistency
1.6 Provisioning
·
transition from one system state to another
system state that's expected and this is generally used in building servers.
·
It’s actually installing and configuring the
software each time rather than just copying the image which is not a clean method.
e.g: On a server having basic operating system, we want to
make it a Web server. Below are the automated steps.
1> Installation
2> Copy configuration
3> Copy web files or some testing codes
4> Install security updates
5> Start the service
1.7 Automation
·
Automation is the most basic part of Puppet
·
Can update files, change network devices, update
the system
E.g:
There is urgent requirement to update a security file on 500
servers ASAP.
Just update the playbook and it will be pushed
automatically. If there is any issue with a particular server, remove it from
load balancer and investigate the cause.
1.8 Orchestration
·
It’s different from automation. Automation used
on a single system whereas orchestration takes the automation and coordinates
the process between multiple systems.
·
Cannot have orchestration without automation.
1.9 Advantages
1.10 Architecture
·
Usually deployed in a simple client-server model
·
Server = Puppet Master (Linux Server), Puppet Master application is
written in Ruby on Rails
·
Client = Nodes
·
Master and Nodes communicate via SSL over
TCP/8140
·
Nodes pull config from Master
·
Each individual configuration item in a Puppet
program is called a resource declaration
Chapter 2
Installing and Configuring the Puppet Master
2.1 Lab Setup
Master – Redhat 7
Client1- CentOS 7
Client2- Ubuntu 14.04.1 LTS
2.2 Installation
·
Install OracleVirtual Box and Vagrant
·
Install CentOS and Ubuntu
2.3 Booting the PuppetMaster
Steps:
1>
Add the CentOS Box to Vagrant
Go to the directory where CentOS
$cd
puppet
$vagrant
box add centos65-base centos65.box
#to verify the box is added
$vagrant
box list
2>
Booting the Puppet Master Server
$cd
puppetmaster
$vagrant
up
3>
Installing Nano, Git and NTP
$sudo
yum –y install nano git ntp
$sudo
service ntpd start
$sudo
chkconfig ntpd on
$chkconfig
| grep ntpd
2.4 Installing the PuppetMaster
Add the puppet repository: to download puppet binaries.
$sudo yum –y install
https://yum.puppetlabs.com/puppetlabs-release-el-7.noarch.rpm
$sudo yum –y install
puppet-server
To check the version of Puppet Master
#puppet master –version
2.5 Setting Up Directory Environments
Directory Env allow to maintain various env like test, dev
and prod on same puppet master.
Puppet Loads the site manifest by Default (pp = Puppet
Program)
/etc/puppet/manifests/site.pp
Tasks:
1>
Create a prod env
2>
Lower the env timeout
3>
Set DNS alternative names
Environment Configuration file:
/etc/puppet/environments/production/environment.conf
Three Section /etc/puppet/puppet.conf
( file exists on both master and agent node)
1.
Master
2.
Agent
3.
Main
[main]
logdir =
/var/log/puppet
rundir =
/var/run/puppet
ssldir =
$vardir/ssl
[master]
environmentpath =
$confdir/environments
basemodulepath =
$confdir/modules:/opt/puppet/share/puppet/modules
[agent]
classfile =
$vardir/classes.txt
localconfig =
$vardir/localconfig
Creating directory
structure
#mkdir –p
/etc/puppet/environments/production/{modules,manifests}
#cd /etc/puppet/environments/production
#vi environment.conf
modulepath =
/etc/puppet/environments/production/modules
environment_timeout = 5s
#cd /etc/puppet/
#vi puppet.conf
( there is no master section : create it)
[main]
……
dns_alt_names =
puppet, puppetmaster, puppetmaster.company.com
[master]
environmentpath = $confdir/environments
basemodulepath
= $confdir/modules:/opt/puppet/share/modules/
[agent]
….
2.6 Security: Dealing with SELinux
Put it into permissive
mode.
#setenforce permissive
File: /etc/sysconfig/selinux
2.7 Puppet Master: Generating Certificates
To generate Certificates: need to perform only one time
#puppet master –verbose –no-daemonize
Break it ^c
#ls –la
/var/lib/puppet/ssl
Suppose we want to re-generate the keys if the current ones
are compromised. Delete ssl (rm –r /var/lib/puppet/ssl)
2.8 Configuring the IPtables Firewalls
IPTables Blocks everything except ICMP and Secure Shell.
Permit other nodes to connect PuppetMaster at TCP/8140.
File Path = /etc/sysconfig/iptables
#vi /etc/sysconfig/iptables
-A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
-A INPUT -p icmp -j ACCEPT
-A INPUT -i lo -j ACCEPT
-A INPUT -m state --state NEW -m tcp -p tcp --dport 22 -j
ACCEPT
-A INPUT -m state --state NEW -m tcp -p tcp --dport 8140
-j ACCEPT
#service iptables restart
Chapter 3
Installing and Configuring the Puppet Agent
3.1 Boot the client1 and client2 server
Add the Ubuntu image in Vagrant on Windows CMD in the
image directory
$cd
$vagrant box add trusty64 trusty-server-clouding-amd64-vagrant-disk1.box
Go to the directory where vagrant file is there and run
below
$vagrant up
Both client will be up
3.2 Adding the PuppetMaster to the hosts file
On both client:
#echo 192.168.56.201 puppetmaster >> /etc/hosts
3.3 Installing the Puppet Agent on CentOS
#yum –y install http://yum.puppetlabs.com/puppetlabs-release-el-6.noarch.rpm
#yum –y install puppet
Now open the /etc/puppet/puppet.conf file, without server
entry puppet agent will not function.
#vi /etc/puppet/puppet.conf
[agent]
server = puppetmaster
3.4 Installing the Puppet Agent on Ubuntu
#wget https://apt.puppetlabs.com/puppetlabs-release-trusty.deb
#dpkg -i puppetlabs-release-trusty.deb
#apt-get update
#apt-get install puppet
Now open the /etc/puppet/puppet.conf file, without server
entry puppet agent will not function.
#vi /etc/puppet/puppet.conf
[agent]
server = puppetmaster
Enable the agent, required in Ubuntu.
#puppet agent --enable
3.5 Generating Certificate from Client
On both client nodes run below command:
#puppet agent --verbose --no-daemonize –onetime
3.6 Signing Agent Certificates
PuppetMaster by default don’t accept the certificates
#puppet cert list
# puppet cert sign client1
# puppet cert sign client2
#puppet cert list all
Chapter 4
Puppet Concept
4.1 Resource Declaration
Each individual configuration item in a Puppet program is
called a resource declaration.
To manage resources on a target system, you should declare
them in Puppet manifests.
Resource Declaration answer two basic questions:
·
What aspect – user, service , package, file
·
What state – stop, start
******************************
type { title:
attribute => value,
}
******************************
4.2 Package Resource
To browse the resource package, run below command:
*************************************
# puppet resource package
package { 'GeoIP':
ensure =>
'1.5.0-11.el7',
}
package { 'NetworkManager':
ensure =>
'1.4.0-17.el7_3',
}
package { 'NetworkManager-libnm':
ensure =>
'1.4.0-17.el7_3',
}
package { 'NetworkManager-team':
ensure =>
'1.4.0-17.el7_3',
}
************************************************************
Package resource Type:
package { ‘ntp’:
ensure =>
‘installed’,
}
4.3 File Resource
Description:
Manages files, including their content, ownership, and
permissions.
File contents can be managed directly with
the content attribute, or downloaded from a remote source using
the source attribute
file { ‘/readme.txt’:
ensure =>
‘present’,
content =>
“Comments are here”,
}
Run below command to check the various attributes of file
/etc.
******************************************************
# puppet resource file /etc
file { '/etc':
ensure => 'directory',
ctime => '2017-03-17 09:11:06 -0400',
group => '0',
mode => '755',
mtime => '2017-03-17 09:10:43 -0400',
owner => '0',
selrange =>
's0',
selrole => 'object_r',
seltype => 'etc_t',
seluser => 'system_u',
type => 'directory',
}
*****************************************************
4.4 Service Resource
Manage running services
e.g:
service { ‘ntpd’:
ensure =>
‘running’ ,
enable => true
, #(note: Boolean attributes shouldn’t
be in quotes, in quotes it takes all true)
}
Run below command to check all the services on the server
******************************************************
# puppet resource service
service { 'NetworkManager-dispatcher.service':
ensure =>
'stopped',
enable =>
'true',
}
service { 'NetworkManager-wait-online.service':
ensure =>
'running',
enable =>
'false',
}
service { 'NetworkManager.service':
ensure =>
'running',
enable =>
'true',
}
*****************************************************************
4.5 Configuration Run
The process of running a Puppet program and applying the
desired configuration.
1>
the node connects to the Puppet Master on TCP
port 8140
2>
the node then sends facts about itself
3>
the Master has to classify the node (means
finding the matching node definition in the Puppet manifest)
4>
the Master then takes this information along
with the facts sent from the node, and it compiles a catalog
5>
the catalog contains two things, the desired
state of each resource and any dependencies the resource requires
6>
the Master sends the catalog to the node
7>
the node then applies the catalog, enforcing the
desired configuration
8>
reports the results to the master
9>
the configuration run is complete
4.6 Facter
Facter is a standalone tool that house’s environment-level
variables.
Each key-value in facter is referred to as “facts”
Command:
# facter
architecture => x86_64
augeasversion => 1.4.0
bios_release_date => 12/01/2006
bios_vendor => innotek GmbH
bios_version => VirtualBox
blockdevice_sda_model => VBOX HARDDISK
blockdevice_sda_size => 8589934592
blockdevice_sda_vendor => ATA
blockdevice_sr0_model => CD-ROM
blockdevice_sr0_size => 1073741312
blockdevice_sr0_vendor => VBOX
blockdevices => sda,sr0
boardmanufacturer => Oracle Corporation
boardproductname => VirtualBox
boardserialnumber => 0
facterversion => 2.4.6
……….
Chapter 5
Creating Manifests
·
Puppet programs are stored in files called
manifests.
·
A manifest is just a regular text file with a
.pp extension that contains Puppet code.
·
A Puppet manifest consists of various elements
such as node definitions, resource declarations, variables, and classes that
control what configuration get applied to what nodes.
·
full path of directory will be
/etc/puppet/environments/production/manifests
·
Puppet will automatically load all of the .pp
files in this directory
5.1 Node Definitions
#vi
/etc/puppet/environments/production/manifests/nodes.pp
node ‘client1’ {
}
node ‘client2’{
}
5.2 Managing Files
·
Earlier we looked at three common resource types
for managing: packages, files, and services.
·
to incorporate these resource types into our new
manifest
Create a simple text file on each server that says,
"Created by Puppet", and then appends the date and time that the file
was created
Make the entry in
nodes.pp
file { '/info.txt':
ensure => 'present',
content =>
inline_template("Created by Puppet at <%= Time.now %>\n"),
}
5.3 Client Firebucket
·
Before Puppet modifies or deletes a file, it
stores the old copy in what's called the client Filebucket
·
Client Filebucket is stored on the node, not on
the Puppet Master
·
File : /var/lib/puppet/clientbucket
·
For restoring the file, hash is required. If we
don’t know than it can be obtained from /var/log/messages
Command:
# puppet filebucket -l --bucket /var/lib/puppet/clientbucket
restore /info.txt 741ed14b73948aecb650c9c368926ff1
Here:
“l” : local means the file is on local server.
Bucket: The full path of clientbucket directory
Last entry: hash value from /var/log/messages
If we just want to know the content but don’t want to restore: replace restore option with
get in above command
# puppet filebucket -l --bucket /var/lib/puppet/clientbucket
get /info.txt 741ed14b73948aecb650c9c368926ff1
5.4 Managing Packages
On puppetmaster: nodes.pp
package { 'ntp':
ensure =>
'installed',
}
On Client: CentOS
# puppet agent --verbose --no-daemonize –onetime
Once the ntp is installed, verify:
CentOS:
# yum info ntp
Installed Packages
Name : ntp
Arch :
x86_64
Ubuntu:
# ls /usr/sbin/ntpd -la
-rwxr-xr-x 1 root root 671440 Oct 5 12:35 /usr/sbin/ntpd
Now the package is installed. Will now have to make sure
it runs.
5.5 Managing Services
On Puppetmaster:
service { 'ntpd':
ensure => 'running',
enable => true,
}
Please note that true will not come under quotes.Also,
the service name in Ubuntu is ntp. So, we have different service name. Will
check later how to deal with this.
On client run below command:
# puppet agent --verbose --no-daemonize –onetime
Now we delete the ntp package and run again the puppet
agent command:
Here, puppet determined that first the package need to be
installed and after that only services can be brought up. However, there are
chances that it can get it wrong. Will see how to take care of this later.
5.6 Selectors
·
On our client1 server which runs CentOS, the ntp
service is called ntpd.
·
On the client2 Ubuntu server it's called ntp.
·
We had to specify separate resource declarations
with different titles under each node definition in our manifest
·
Not an efficient way
·
If we are having many different nodes, it
becomes unmanageable.
·
To make it scalable and readable by using a
Puppet feature called a selector
·
Uses variables
e.g:
$ntpservice = $osfamily ? {
‘redhat’ => ‘ntpd’,
‘debian’ => ‘ntp’,
default =>
‘ntp’,
}
How to use it:
Replace:
service { 'ntpd':
ensure => 'running',
enable => true,
}
With:
service {
$ntpservice:
ensure => 'running',
enable => true,
}
5.7 Classes
·
At this point, we have a lot of repetition in
our code.
·
Issues: inefficient, doesn’t scale very
well, difficult to read and understand.
·
Should not repeat the code.
·
Class can be used to clean up the code and
arrange it better.
·
Puppet Class is very different from Object
Oriented Class.
·
A Puppet class is simply a named collection of
resources.
·
Adv:
class is that you can write it once and then reuse it throughout the Puppet
programs
e.g:
class linux {
package { ‘ntp’:
ensure =>
‘installed’,
}
}
node ‘wiki’ {
{ class ‘linux’:}
}
nodes.pp
node 'client1' {
class { 'linux':
}
}
node 'client2' {
class { 'linux':
}
}
class linux {
$ntpservice =
$osfamily ? {
'redhat'
=> 'ntpd',
'debian'
=> 'ntp',
default =>
'ntp',
}
file {
'/info.txt':
ensure => 'present',
content =>
inline_template("Created by Puppet at <%= Time.now %>\n"),
}
package { 'ntp':
ensure =>
'installed',
}
service {
$ntpservice :
ensure => 'running',
enable => true,
}
}
5.8 Using Variables
We can use variables in another way to avoid repetition.
Suppose we have three packages we want to install, Nano,
Git, and Screen.
Instead of writing resource for each, we can use variables
to write it once.
class linux {
$packlist =
['git', 'nano', 'screen']
package {
$packlist:
ensure =>
'installed',
}
…….
Old Entries
……..
}
Package might not installed in the order in which they are
specified.
Chapter 6
Creating and Using Modules
6.1 Definition
Modules are directories with a predictable structure.
Puppet can automatically load manifests, files, and plugins
from modules in its modulepath.
6.2 Finding Modules
Use puppet master --configprint modulepath to see where
Puppet expects to find modules on your system.
# puppet master --configprint modulepath
/etc/puppet/environments/production/modules
6.3 Creating Modules
#cd /etc/puppet/environments/production/modules/
#puppet module generate yourname-mediawiki –environment
production
#mv yourname-mediawiki mediawiki
#cd mediawiki/manifests
Check the file init.pp. It is having one class called
mediawiki.
6.4 Module Structure
Collection of Puppet code organized into a predictable file
structure.
Parent Directory is having below 7 Directories
·
Manifests
·
Files
·
Templates
·
Lib
·
Facts.d
·
Tests
·
Spec
6.5 Manifests
Puppet code unique to the module, including classes,
resource declarations, and variables
Every Puppet module contains an init.pp file in the manifest
directory.
The init.pp file, has an empty class declaration for the
mediawiki class.
The init.pp manifest is automatically loaded this means is
that we can actually invoke this mediawiki class from inside our nodes.pp
manifest the same way we invoked our linux class earlier.
6.6 Files
Contains files that we want to push to the Puppet nodes usually
configuration files.
E.g: IPtables, SELinux.
This is for only static files that don't change.
For dynamic files, will use next directory Templates.
6.7 Templates
This directory holds ERB templates.
We can have entire file template that contains a mix of
static and dynamic content that we can modify or customize with variables and
facts
Use the template function to create a string by rendering a
template.
Use the content attribute to fill file contents with a
string.
Template files are referenced as modulename/filename.erb
**************************************************************
Apache/templates/vhost.erb
To use this template:
file {'/etc/apache2/sites-enabled/wordpress.conf':
ensure => file,
content =>
template('apache/vhost.erb'),
}
***************************************************************
6.8 Lib
Contains custom facts.
Custom facts require to write Ruby code.
6.9 Facts.d:
Contains external facts.
These are similar to custom facts, except instead of using
Ruby to write custom facts, we can use any script or executable.
6.10 Tests and spec:
For doing unit testing and requires some knowledge of Ruby.
6.11 Customising Module
Create below entry inside class mediawiki
# cat
/etc/puppet/environments/production/modules/mediawiki/manifests/init.pp
class mediawiki {
$phpmysql = $osfamily ? {
'redhat' =>
'php-mysql',
'debian' =>
'php5-mysql',
default => 'php-mysql',
}
package {
$phpmysql:
ensure =>
'present',
}
}
6.12 How to use class from a module inside a node definition
Now include above class inside each node definition in
nodes.pp
# cat
/etc/puppet/environments/production/manifests/nodes.pp
node 'client1' {
class { 'linux':
}
class { 'mediawiki': }
}
node 'client2' {
class { 'linux':
}
class { 'mediawiki': }
}
class linux {
$packlist =
['git', 'nano', 'screen']
package {
$packlist:
ensure =>
'installed',
}
$ntpservice =
$osfamily ? {
'redhat'
=> 'ntpd',
'debian'
=> 'ntp',
default =>
'ntp',
}
file {
'/info.txt':
ensure => 'present',
content =>
inline_template("Created by Puppet at <%= Time.now %>\n"),
}
package { 'ntp':
ensure =>
'installed',
}
service {
$ntpservice :
ensure => 'running',
enable => true,
}
}
Run the agent command on both server client1 and client2.
# puppet agent --verbose --no-daemonize –onetime
Ubuntu:
# service --status-all | grep -i mysql
6.13 Conditionals
If we want to install a package only on CentOS but not on
Ubuntu then we use conditionals.
Add this in init.pp inside mediawiki class.
if $osfamily ==
'redhat' {
package {
'php-xml':
ensure =>
'present',
}
}
Run the agent command on both server client1 and client2.
This package php-xml got installed only on client1.
# puppet agent --verbose --no-daemonize –onetime
CentOS:
# systemctl list-unit-files | grep -i mysql
When you run:
#yum install mysql
command by default it installs MariaDB not MySQL.
So try the following command:
#yum list installed | grep mariadb
if mariadb-server is missing try this following command:
#yum install mariadb-server
it installs the server package then start the service:
#systemctl start mariadb
Or create below entries in nodes.pp
***************************************************
$packlist = ['nano', 'screen', 'mariadb', 'mariadb-server']
package { $packlist:
ensure =>
'installed',
}
if $osfamily == 'redhat' {
service { 'mariadb':
ensure => 'running',
}
}
*********************************************
Chapter 7
Project
7.1 The Apache Module
Install the Apache Module, run below command on puppetserver
# puppet module install puppetlabs-apache --modulepath /etc/puppet/environments/production/modules/
Go to below link:
Make below entries inside mediawiki class in init.pp inside
mediawiki manifests
class mediawiki {
$phpmysql = $osfamily ? {
'redhat' =>
'php-mysql',
'debian' =>
'php5-mysql',
default => 'php-mysql',
}
package {
$phpmysql:
ensure =>
'present',
}
if $osfamily ==
'redhat' {
package {
'php-xml':
ensure =>
'present',
}
}
class { '::apache':
docroot
=> '/var/www/html',
mpm_module => 'prefork',
subscribe
=> Package[$phpmysql],
}
class { '::apache::mod::php' :}
}
Run the agent command on both server client1 and client2.
This package php-xml got installed only on client1.
# puppet agent --verbose --no-daemonize –onetime
Check apache is running
CentOS:
#service httpd status
Ubuntu:
#service apache2 status
7.2 The vcsrepo Module
We need to fetch the mediawiki application files from
mediawiki's Github repository and place them in Apache's document root
directory, which is /var/www/html.
Files are located at github.com/wikimedia/mediawiki.git, but
Puppet does not have a built-in method to use Git.
Vcsrepo module will simply use Git to copy down the
mediawiki files and put them wherever we want.
Install the vcsrepo:
# puppet module install puppetlabs-vcsrepo --modulepath /etc/puppet/environments/production/modules/
On below link:
Update Mediawiki class inside init.pp.
class mediawiki {
$phpmysql = $osfamily ? {
'redhat' =>
'php-mysql',
'debian' =>
'php5-mysql',
default => 'php-mysql',
}
package {
$phpmysql:
ensure =>
'present',
}
if $osfamily ==
'redhat' {
package {
'php-xml':
ensure =>
'present',
}
}
class { '::apache':
docroot => '/var/www/html',
mpm_module
=> 'prefork',
subscribe => Package[$phpmysql],
}
class {
'::apache::mod::php' :}
vcsrepo { '/var/www/html':
ensure
=> 'present',
provider => 'git',
source
=> "https://github.com/wikimedia/mediawiki.git",
revision => 'REL1_23',
}
}
Run the agent command on both server client1 and client2.
# puppet agent --verbose --no-daemonize –onetime
It failed on Ubuntu as there is a file index.html inside
/var/www/html. So, we need to make sure it get deleted.
Add below entry in init.pp on puppetmaster.
********************************
file {
'/var/www/html/index.html':
ensure => 'absent',
}
*********************************
It failed again as
the order was not correct. Before the
file get deleted, it started to download the file from github.
In next chapter will see how to take care of ordering.
7.3 Resource Ordering
Ensure that Puppet deletes /var/www/html/index.html file
before invoking vcsrepo.
Can be achieved using an ordering row.
Ordering Row Syntax:
File[‘/var/www/html/index.html’] ->
Vcsrepo[‘/var/www/html’]
Init.pp
class mediawiki {
$phpmysql = $osfamily ? {
'redhat' =>
'php-mysql',
'debian' =>
'php5-mysql',
default => 'php-mysql',
}
package {
$phpmysql:
ensure =>
'present',
}
if $osfamily ==
'redhat' {
package {
'php-xml':
ensure =>
'present',
}
}
class {
'::apache':
docroot => '/var/www/html',
mpm_module
=> 'prefork',
subscribe => Package[$phpmysql],
}
class {
'::apache::mod::php' :}
vcsrepo {
'/var/www/html':
ensure => 'present',
provider =>
'git',
source =>
"https://github.com/wikimedia/mediawiki.git",
revision =>
'REL1_23',
}
file {
'/var/www/html/index.html':
ensure => 'absent',
}
File['/var/www/html/index.html'] ->
Vcsrepo['/var/www/html']
}
Create the file as it was deleted during last run
#touch /var/www/html/index.html
Run the agent command on both server client1 and client2.
# puppet agent --verbose --no-daemonize –onetime
This time it ran successfully.
7.4 The MySQL Module
Install the Mysql module
# puppet module install puppetlabs-mysql --modulepath /etc/puppet/environments/production/modules/
Go to the link
Make below entry in file /etc/puppet/environments/production/modules/mediawiki/manifests/init.pp
class mediawiki {
$phpmysql = $osfamily ? {
'redhat' =>
'php-mysql',
'debian' =>
'php5-mysql',
default => 'php-mysql',
}
package {
$phpmysql:
ensure =>
'present',
}
if $osfamily ==
'redhat' {
package { 'php-xml':
ensure =>
'present',
}
}
class {
'::apache':
docroot => '/var/www/html',
mpm_module
=> 'prefork',
subscribe => Package[$phpmysql],
}
class {
'::apache::mod::php' :}
vcsrepo {
'/var/www/html':
ensure => 'present',
provider =>
'git',
source =>
"https://github.com/wikimedia/mediawiki.git",
revision =>
'REL1_23',
}
file {
'/var/www/html/index.html':
ensure => 'absent',
}
File['/var/www/html/index.html'] -> Vcsrepo['/var/www/html']
class { '::mysql::server':
root_password =>
'training',
}
}
Run the agent command on both server client1 and client2.
# puppet agent --verbose --no-daemonize --onetime
Ubuntu:
# service mysql status
mysql start/running, process 4546
# mysql -u root -p
Enter password:training
Welcome to the MySQL monitor. Commands end with ; or \g.
mysql> \q
Bye
7.5 The Firewall Module
Install the firewall module
# puppet module install puppetlabs-firewall --modulepath
/etc/puppet/environments/production/modules/
Make below entry in file /etc/puppet/environments/production/modules/mediawiki/manifests/init.pp
class mediawiki {
$phpmysql = $osfamily ? {
'redhat' =>
'php-mysql',
'debian' =>
'php5-mysql',
default => 'php-mysql',
}
package {
$phpmysql:
ensure =>
'present',
}
if $osfamily ==
'redhat' {
package {
'php-xml':
ensure =>
'present',
}
}
class {
'::apache':
docroot => '/var/www/html',
mpm_module
=> 'prefork',
subscribe => Package[$phpmysql],
}
class {
'::apache::mod::php' :}
vcsrepo {
'/var/www/html':
ensure => 'present',
provider =>
'git',
source =>
"https://github.com/wikimedia/mediawiki.git",
revision =>
'REL1_23',
}
file {
'/var/www/html/index.html':
ensure => 'absent',
}
File['/var/www/html/index.html'] -> Vcsrepo['/var/www/html']
class {
'::mysql::server':
root_password => 'training',
}
class { '::firewall' : }
firewall { '000 allow http access':
port
=> '80',
proto
=> 'tcp',
action => 'accept',
}
}
Ubuntu:
#iptables -L
Now go to browser and type the IP address of clients
Go through various option and complete the installation.
When we visited the MediaWiki site on the wiki server,
MediaWiki made us go through a configuration process that generated a
configuration settings file, which had then asked us to download, save, and
then re-upload to the server. Now we need to automate this. We will achieve
this by using File templates.
Database name: wiki
Database username: root
Database password: training
Database name: wiki
Name of wiki: wiki
Your username: root
Password: training
Chapter 8
Creating File Templates
8.1 Introduction
File templates allow
us to take a static file like a configuration file and insert dynamic content
into it wherever we want.
The file MediaWiki gave us to download was called
LocalSettings.php
This file contains configuration like the wiki Sitename, the
MySQL database name, the root password, and so on.
Most of this information, which sits on the right side of
the equal signs, is information that we put in manually during the MediaWiki
setup process.
In other words, everything to the right of the equal signs
is dynamic content.
We will use puppet to fill this dynamic content and generate
a valid useable LocalSettings.php file using a file template.
A template is a regular text file with a .erb extension
(embedded Ruby block).
ERB - starting with the <%=, and ending with a %> sign
8.2 Creating File Templates
Will start with working configuration file like
LocalSettings.php file.
We need to take the dynamic content areas that you want
Puppet to manage, and replace those with embedded Ruby blocks.
$wgSitename = "wiki";
$wgMetaNamespace = "Wiki";
$wgServer = "http://192.168.56.203";
$wgDBserver = "localhost";
$wgDBname = "wiki";
$wgDBuser = "root";
$wgDBpassword = "training";
$wgUpgradeKey = "9e7812b95263dc6c";
Left Hand Side= All PHP variable ( the file
LocalSettings.php is php file)
These variables are used by mediawiki, puppet is not having
any information about these.
We're going to replace the values on the right of the =
sign.
LocalSettings.erb
$wgSitename = "<%= wikisitename %>";
$wgMetaNamespace = "<%= wikimetanamespace %>";
$wgServer = "<%= wikiserver %>";
$wgDBserver = "<%= wikidbserver %>";
$wgDBname = "<%= wikidbname %>";
$wgDBuser = "<%= wikidbuser %>";
$wgDBpassword = "<%= wikidbpassword %>";
$wgUpgradeKey = "<%= wikiupgradekey %>";
Replace above variables with right side value in the
original file.
As MediaWiki module is going to use this template, will
place original file under template directory.
#mkdir /etc/puppet/environments/production/modules/mediawiki/templates
Copy the file inside above directory.
8.3 Using Templates in Modules
Template takes a file with a mix of static and dynamic
content and uses it to generate a new file
Here we want our template to generate the LocalSettings.php
file.
MediaWiki told us to place the LocalSettings.php file in the
root of the MediaWiki installation which is /var/www/html on wiki server.
We will use the file resource type to use our template.
This file resource will go to MediaWiki modules init.pp
manifest, not the nodes manifest.
/etc/puppet/environments/production/modules/mediawiki/manifests/init.pp
file { ‘LocalSettings.php’ :
path =>
‘/var/www/html/LocalSettings.php’,
ensure => ‘file’,
content =>
template(‘mediawiki/LocalSettings.erb’),
}
The template uses Puppet variables to populate portions of
the configuration file.
These variables are defined in nodes.pp file.
/etc/puppet/environments/production/manifests/nodes.pp
node 'client2' {
$wikisitename = 'wiki'
$wikimetanamespace = 'Wiki'
$wikiserver = "http://192.168.56.203
$wikidbserver = 'localhost'
$wikidbname = 'wiki'
$wikidbuser = 'root'
$wikidbpassword = 'training'
$wikiupgradekey = 'puppet'
class { 'linux':
}
class {
'mediawiki': }
}
Add below entry in init.pp file inside mediawiki class.
file { 'LocalSettings.php':
path
=> '/var/www/html/LocalSettings.php',
ensure
=> 'file',
content =>
template("mediawiki/LocalSettings.erb"),
}
Verification:
Go to browser and tyoe the IP address of client. It
should falg message that mediawiki is successfully installed.
Chapter 9
Configuring Hiera
In last chapter, we observed that the variables assigned
under our wiki node definition called node scope.
Drawback: We have to keep all of these variables in our
nodes manifest under each and every node definition.
We will address this problem by using Hiera.
Advantage of Hiera
·
Provides a separate place to store node specific
data.
·
Not tied to any module or manifest –
independent.
·
Help minimize the repetition.
9.1 Using Hiera Data in Modules
The best place for module-specific variable assignments is
in the module itself.
/etc/puppet/environments/production/modules/mediawiki/manifests/init.pp
$wikimetanamespace =
hiera('mediawiki::wikimetanamespace')
$wikisitename = hiera('mediawiki::wikisitename')
$wikiserver =
hiera('mediawiki::wikiserver')
$wikidbserver = hiera('mediawiki::wikidbserver')
$wikidbuser = hiera('mediawiki::wikidbuser')
$wikidbpassword = hiera('mediawiki::wikidbpassword')
$wikiupgradekey = hiera('mediawiki::wikiupgradekey')
Explanation:
mediawiki::wikimetanamespace -> This is called Key
Key value pair is similar to variable assignment. It's
just that instead of setting a Puppet variable equal to a particular value, you
set a key equal to a particular value.
When Puppet encounters a hiera function call with a key,
it does a look-up on that key to retrieve the value associated with it.
It goes and look into a text file in YAML format.
When used with Hiera, these YAML files are called data
sources.
If there are many YAML files and they are containing overlapping keys than
which value does puppet consider. For this it checks the Hiera configuration
file, which is located, by default, at /etc/puppet/hiera.yaml.
*********************
:backends:
- yaml
:yaml:
:datadir:
:hierarchy:
-
"%{clientcert}"
- clientdefault
**********************
First Two Lines:
Means we're going to use YAML for our Hiera data, as opposed to a different
type of data structure.
Next Two Lines: Indicates
where our YAML files are stored. Here there's no directory listed so it's going
to use the default, which is /var/lib/hiera.
Last Part Hierarchy: To
decide which file to use.When Puppet does a Hiera look-up, it starts at the top
of this list and checks the first file. Clientcert is a variable containing the
hostname of the node that runs the Puppet agent. In our case its client2, so it
will look for client2.yaml and it will check that for Hiera data. It will
search for this file in default
directory - /var/lib/hiera. If puppet
don’t find the client2.yaml file it will go down to clientdefault and check the
clientdefault.yaml.
9.2 Creating YAML Data Sources
The .yaml files which
will store the key value pairs for our MediaWiki module are called data
sources.
Will create three data sources, client1.yaml, client2.yaml
and clientdefault.yaml inside /var/lib/hiera.
Create below three files:
/var/lib/hiera/client2.yaml
---
mediawiki::wikisitename: wiki
mediawiki::wikimetanamespace: Wiki
mediawiki::wikiserver: http://192.168.56.203
mediawiki::wikidbname: wiki
/var/lib/hiera/client3.yaml
---
mediawiki::wikisitename: wiki
mediawiki::wikimetanamespace: Wiki
mediawiki::wikiserver: http://192.168.56.204
mediawiki::wikidbname: wiki
/var/lib/hiera/clientdefault.yaml
---
mediawiki::wikidbserver: localhost
mediawiki::wikidbuser: root
mediawiki::wikidbpassword: training
mediawiki::wikiupgradekey: puppet
9.3 Assigning Classes in Hiera
We still have a couple of class declarations being repeated.
/etc/puppet/environments/production/manifests/nodes.pp
node 'client2' {
hiera_include('classes')
}
node 'client3' {
hiera_include('classes')
}
/var/lib/hiera/client2.yaml
---
classes:
- mediawiki
- linux
mediawiki::wikisitename: wiki
mediawiki::wikimetanamespace: Wiki
mediawiki::wikiserver: http://192.168.56.203
mediawiki::wikidbname: wiki
/var/lib/hiera/client3.yaml
---
classes:
- mediawiki
- linux
mediawiki::wikisitename: wiki
mediawiki::wikimetanamespace: Wiki
mediawiki::wikiserver: http://192.168.56.204
mediawiki::wikidbname: wiki
Enabling Hiera
Restart puppetmaster enable puppet to pick up the changes
made to the hiera.yaml configuration file.
Chapter 10
Version Control
Version control allows us to take snapshots of our Puppet
configuration, including our configuration files, manifests, modules, and Hiera
data.
Can recover easily if we make changes that breaks the
puppet.
10.1 Automating the Puppet Agent
Till now we were running the puppet agent manually.
In a production environment it is designed to run
automaticall.
Frequency of the configuration run is called Run Interval.
By default Run Interval = 30 Min.
To change default Run Interval: On each client server edit below file.
/etc/puppet/puppet.conf
*******************
[main]
runinterval=60
*******************
To make the
agents run automatically once the server boots up.
CentOS 6:
#chkconfig puppet
on
#service puppet
start
CentOS 7:
#systemctl enable puppet
#systemctl start puppet
Ubuntu:
File: /etc/default/puppet
START=yes
#service puppet start
10.2 Setup a Git Repository
git config --global user.name "XYZ"
git config --global user.email
"XYZ@gmail.com"
cd /etc/puppet/environments/production/
git init
git add .
git commit -m "Initial Commit"
