Salt stack issues

  • The function “state.apply” is running as PID

Restart salt-minion with command: service salt-minion restart

  • No matching sls found for ‘init’ in env ‘base’

Add top.sls file in the directory where your main sls file is present.

Create the file as follows:

base:
'web*':
- apache

If the sls is present in a subdirectory elasticsearch/init.sls then write the top.sls as:

base:
'*':
- elasticsearch.init
  • How to execute saltstack-formulas
    1. create file /srv/pillar/top.sls with content:
    base:
      '*':
        - salt
    1. create file /srv/pillar/salt.sls with content:
    salt:
      master:
        worker_threads: 2
        fileserver_backend:
          - roots
          - git
        gitfs_remotes:
          - git://github.com/saltstack-formulas/epel-formula.git
          - git://github.com/saltstack-formulas/git-formula.git
          - git://github.com/saltstack-formulas/nano-formula.git
          - git://github.com/saltstack-formulas/rabbitmq-formula.git
          - git://github.com/saltstack-formulas/remi-formula.git
          - git://github.com/saltstack-formulas/vim-formula.git
          - git://github.com/saltstack-formulas/salt-formula.git
          - git://github.com/saltstack-formulas/users-formula.git
        external_auth:
          pam:
            tiger:
              - .*
              - '@runner'
              - '@wheel'
        file_roots:
          base:
            - /srv/salt
        pillar_roots:
          base:
            - /srv/pillar
        halite:
          level: 'debug'
          server: 'gevent'
          host: '0.0.0.0'
          port: '8080'
          cors: False
          tls: True
          certpath: '/etc/pki/tls/certs/localhost.crt'
          keypath: '/etc/pki/tls/certs/localhost.key'
          pempath: '/etc/pki/tls/certs/localhost.pem'
      minion:
        master: localhost
    1. before you can use saltstack-formula you need to make one change to /etc/salt/master and add next config:
    fileserver_backend:
      - roots
      - git
    gitfs_remotes:
      - git://github.com/saltstack-formulas/salt-formula.git
    1. restart salt-master (e.g. service salt-master restart)
    2. run salt-call state.sls salt.master
  • The Salt Master has cached the public key for this node

Execute the following command:

delete the exiting key on master by:

salt-key -d <minion-id>

then restart minion. Then reaccept the key on master:

salt-key -a <minion-id>

  • If salt-cloud is giving error as below:

Missing dependency: ‘netaddr’. The openstack driver requires ‘netaddr’ to be installed.

Execute the command: yum install python-netaddr

then verify if your provider is loaded with command: salt-cloud –list-providers

  • Remove dead minions keys in salt

salt-run manage.down removekeys=True

Advertisements

Diamond installation on centos 7

$ yum install make rpm-build python-configobj python-setuptools
$ git clone https://github.com/python-diamond/Diamond
$ cd Diamond
$ make buildrpm
Then use the package you built like this:

$ yum localinstall –nogpgcheck dist/diamond-4.0.449-0.noarch.rpm
$ cp /etc/diamond/{diamond.conf.example,diamond.conf}
$ $EDITOR /etc/diamond/diamond.conf
# Start Diamond service via service manager.

$ service diamond start

diamond-setup -C ElasticSearchCollector
diamond-setup -C NetworkCollector

Issues

  1. failed to connect socket to ‘/var/run/libvirt/libvirt-sock-ro’ no such file or directory

execute: egrep ‘(vmx|svm)’ /proc/cpuinfo

2. If the above commands returns with any output showing vmx or svm then your hardware supports VT else it does not.

yum install qemu-kvm qemu-img virt-manager libvirt libvirt-python libvirt-client virt-install virt-viewer

pyDash – A Web Based Linux Performance Monitoring Tool

pydash is a lightweight web-based monitoring tool for Linux written in Python and Django plus Chart.js. It has been tested and can run on the following mainstream Linux distributions: CentOS, Fedora, Ubuntu, Debian, Arch Linux, Raspbian as well as Pidora.

You can use it to keep an eye on your Linux PC/server resources such as CPUs, RAM, network stats, processes including online users and more. The dashboard is developed entirely using Python libraries provided in the main Python distribution, therefore it has a few dependencies; you don’t need to install many packages or libraries to run it.

In this article, we will show you how to install pydash to monitor Linux server performance.

How to Install pyDash in Linux System

1. First install required packages: git and Python pip as follows:

-------------- On Debian/Ubuntu -------------- 
$ sudo apt-get install git python-pip
-------------- On CentOS/RHEL -------------- 
# yum install epel-release
# yum install git python-pip
-------------- On Fedora 22+ --------------
# dnf install git python-pip

2. If you have git and Python pip installed, next, install virtualenv which helps to deal with dependency issues for Python projects, as below:

# pip install virtualenv
OR
$ sudo pip install virtualenv

3. Now using git command, clone the pydash directory into your home directory like so:

# git clone https://github.com/k3oni/pydash.git
# cd pydash

4. Next, create a virtual environment for your project called pydashtest using the virtualenv command below.

$ virtualenv pydashtest #give a name for your virtual environment like pydashtest

Create Virtual Environment

Important: Take note the virtual environment’s bin directory path highlighted in the screenshot above, yours could be different depending on where you cloned the pydash folder.

5. Once you have created the virtual environment (pydashtest), you must activate it before using it as follows.

$ source /home/aaronkilik/pydash/pydashtest/bin/activate

Active Virtual Environment

From the screenshot above, you’ll note that the PS1 prompt changes indicating that your virtual environment has been activated and is ready for use.

6. Now install the pydash project requirements; if you are curious enough, view the contents of requirements.txt using the cat command and the install them using as shown below.

$ cat requirements.txt
$ pip install -r requirements.txt

7. Now move into the pydash directory containing settings.py or simple run the command below to open this file to change the SECRET_KEY to a custom value.

$ vi pydash/settings.py

Set Secret Key

Save the file and exit.

8. Afterward, run the django command below to create the project database and install Django’s auth system and create a project super user.

$ python manage.py syncdb

Answer the questions below according to your scenario:

Would you like to create one now? (yes/no): yes
Username (leave blank to use 'root'): admin
Email address: aaronkilik@gmail.com
Password: ###########
Password (again): ############

Create Project Database

9. At this point, all should be set, now run the following command to start the Django development server.

$ python manage.py runserver

10. Next, open your web browser and type the URL: http://127.0.0.1:8000/ to get the web dashboard login interface. Enter the super user name and password you created while creating the database and installing Django’s auth system in step 8 and click Sign In.

pyDash Login Interface

11. Once you login into pydash main interface, you will get a section for monitoring general system info, CPU, memory and disk usage together with system load average.

Simply scroll down to view more sections.

pyDash Server Performance Overview

12. Next, screenshot of the pydash showing a section for keeping track of interfaces, IP addresses, Internet traffic, disk read/writes, online users and netstats.

pyDash Network Overview

13. Next is a screenshot of the pydash main interface showing a section to keep an eye on active processes on the system.

pyDash Active Linux Processes

Pyinotify – Monitor Filesystem Changes in Real-Time in Linux

Pyinotify is a simple yet useful Python module for monitoring filesystems changes in real-time in Linux.

As a System administrator, you can use it to monitor changes happening to a directory of interest such as web directory or application data storage directory and beyond.

It depends on inotify (a Linux kernel feature incorporated in kernel 2.6.13), which is an event-driven notifier, its notifications are exported from kernel space to user space via three system calls.

The purpose of pyinotify is to bind the three system calls, and support an implementation on top of them providing a common and abstract means to manipulate those functionalities.

In this article, we will show you how to install and use pyinotify in Linux to monitor filesystem changes or modifications in real-time.

Dependencies

In order to use pyinotify, your system must be running:

  1. Linux kernel 2.6.13 or higher
  2. Python 2.4 or higher

How to Install Pyinotify in Linux

First start by checking the kernel and Python versions installed on your system as follows:

# uname -r 
# python -V

Once dependencies are met, we will use pip to install pynotify. In most Linux distributions, Pip is already installed if you’re using Python 2 >=2.7.9 or Python 3 >=3.4 binaries downloaded from python.org, otherwise, install it as follows:

# yum install python-pip      [On CentOS based Distros]
# apt-get install python-pip  [On Debian based Distros]
# dnf install python-pip      [On Fedora 22+]

Now, install pyinotify like so:

# pip install pyinotify

It will install available version from the default repository, if you are looking to have a latest stable version of pyinotify, consider cloning it’s git repository as shown.

# git clone https://github.com/seb-m/pyinotify.git
# cd pyinotify/
# ls
# python setup.py install

How to Use pyinotify in Linux

In the example below, I am monitoring any changes to the user tecmint’s home (/home/tecmint) directory as root user (logged in via ssh) as shown in the screenshot:

# python -m pyinotify -v /home/tecmint

Monitor Directory Changes

Next, we will keep a watch for any changes to the web directory (/var/www/html/tecmint.com):

# python -m pyinotify -v /var/www/html/tecmint.com

To exit the program, simply hit [Ctrl+C].

Note: When you run pyinotify without specifying any directory to monitor, the /tmp directory is considered by default.

HOW TO INSTALL AND REMOVE SOFTWARE IN UBUNTU

When you switch to Linux, the experience could be overwhelming at the start. Even the basic things like installing applications in Ubuntu could seem confusing.

Don’t worry. Linux provides so many ways to do the same task that it is only natural that you may seem lost, at least in the beginning. You are not alone. We have all been to that stage.

In this beginner’s guide, I’ll show most popular ways to install software in Ubuntu. I’ll also show you how to uninstall the software you had installed earlier.

I’ll also provide my recommendation about which method you should be using for installing software in Ubuntu. Sit tight and pay attention. This is a long article, a detailed one which is surely going to add to your knowledge.

INSTALLING AND UNINSTALLING SOFTWARE IN UBUNTU

I am using Ubuntu 16.04 running with Unity desktop environment in this guide. Apart from a couple of screenshots, this guide is applicable to all other flavors of Ubuntu.

1.1 INSTALL SOFTWARE USING UBUNTU SOFTWARE CENTER [RECOMMENDED]

The easiest and most convenient way to find and install software in Ubuntu is by using Ubuntu Software Center. In Ubuntu Unity, you can search for Ubuntu Software Center in Dash and click on it to open it:

Run Ubuntu Software Center
Run Ubuntu Software Center

You can think of Ubuntu Software Center as Google’s Play Store or Apple’s App Store. It showcases all the software available for your Ubuntu system. You can either search for an application by its name or just browse through various categories of software. You can also opt for the editor’s pick. Your choice mainly.

Installing software in Ubuntu using Ubuntu Software Center

Once you have found the application you are looking for, simply click on it. This will open a page inside Software Center with a description of the application. You can read the description, see its raiting and also read reviews. You can also write a review if you want.

Once you are convinced that you want the application, you can click on the install button to install the selected application. You’ll have to enter your password in order to install applications in Ubuntu.

Installing software in Ubuntu: The easy way
Check details and then install software

Can it be any easier than this? I doubt that.

//giphy.com/embed/l4JyIRMDFnC3cM54k?html5=true

Tip: As I had mentioned in things to do after installing Ubuntu 16.04, you should enable Canonical partner repository. By default, Ubuntu provides only those softwares that come from its own repository (verified by Ubuntu).

But there is also a Canonical Partner repository which is not directly controlled by Ubuntu and includes closed source proprietary software. Enabling this repository gives you access to more software. Installing Skype in Ubuntu is achieved by this method.

In Unity Dash, look for Software & Updates.

Ubuntu Software Update Settings

And in here, under Other Software tab, check the options of Canonical Partners.

Enable Canonical partners in Ubuntu 14.04
Enable Canonical partners to access more software

1.2 REMOVE SOFTWARE USING UBUNTU SOFTWARE CENTER [RECOMMENDED]

We just saw how to install software using Ubuntu Software Center. How about removing software that you had installed using this method?

Uninstalling software with Ubuntu Software Center is as easy as the installation process itself.

Open the Software Center and click on the Installed tab. It will show you all the installed software. Alternatively, you can just search for the application by its name.

To remove the application from Ubuntu, simply click on Remove button. Again you will have to provide your password here.

Uninstall software installed in Ubuntu
Find installed software and remove them

2.1 INSTALL SOFTWARE IN UBUNTU USING .DEB FILES

.deb files are similar to the .exe files in Windows. This is an easy way to provide software installation. Many software vendors provide their software in .deb format. Google Chrome is such an example.

You can download .deb file from the official website.

Downloading deb packaging

Once you have downloaded the .deb file, just double click on it to run it. It will open in Ubuntu Software Center and you can install it in the same way as we saw in section 1.1.

Alternatively, you can use a lightweight program Gdebi to install .deb files in Ubuntu.

Once you have installed the software, you are free to delete the downloaded .deb file.

Tip: A few things to keep in mind while dealing with .deb file.

  • Make sure that you are downloading the .deb file from the official source. Only rely on the official website or GitHub pages.
  • Make sure that you are downloading the .deb file for correct system type (32 bit or 64 bit). Read our quick guide to know if your Ubuntu system is 32 bit or 64 bit.

2.2 REMOVE SOFTWARE THAT WAS INSTALLED USING .DEB

Removing software that was installed by a .deb file is the same as we saw earlier in section 1.2. Just go to Ubuntu Software Center, search for the application name and click on remove to uninstall it.

Alternatively, you can use Synaptic Package Manager. Not necessarily but this may happen that the installed application is not visible in Ubuntu Software Center. Synaptic Package Manager is lists all the software that are available for your system and all the software that are already installed on your system.This is a very powerful and very useful tool.

This is a very powerful and very useful tool. Before Ubuntu Software Center came into existence to provide a more user-friendly approach to software installation, Synaptic was the default program for installing and uninstalling software in Ubuntu.

You can install Synaptic package manager by clicking on the link below (it will open Ubuntu Software Center).

Install Synaptic Package Manager

Open Synaptic Manager and then search for the software you want to uninstall. Installed softwares are marked with a green button. Click on it and select “mark for removal”. Once you do that, click on “apply” to remove the selected software.

Using Synaptic to remove software in Ubuntu

3.1 INSTALL SOFTWARE IN UBUNTU USING APT COMMANDS [RECOMMENDED]

You might have noticed a number of websites giving you a command like “sudo apt-get install” to install software in Ubuntu.

This is actually the command line equivalent of what we saw in section 1. Basically, instead of using the graphical interface of Ubuntu Software Center, you are using the command line interface. Nothing else changes.

Using the apt-get command to install software is extremely easy. All you need to do is to use a command like:

sudo apt-get install package_name

Here sudo gives ‘admin’ or ‘root’ (in Linux term) privileges. You can replace package_name with the desired software name.

apt-get commands have auto-completion so if you type a few letters and hit tab, it will provide all the programs matching with those letters.

3.2 REMOVE SOFTWARE IN UBUNTU USING APT COMMANDS [RECOMMENDED]

You can easily remove softwares that were installed using Ubuntu Software Center, apt command or .deb file using the command line.

All you have to do is to use the following command, just replace the package_name with the software name you want to delete.

sudo apt-get remove package_name

Here again, you can benefit from auto completion by pressing the tab key.

Using apt-get commands is not rocket science. This is in fact very convenient. With these simple commands, you get acquainted with the command line part of Ubuntu Linux and it does help in long run. I recommend reading my detailed guide on using apt-get commands to learn in detail about it.

4.1 INSTALL APPLICATIONS IN UBUNTU USING PPA

PPA stands for Personal Package Archive. This is another way that developers use to provide their software to Ubuntu users.

In section 1, you came across a term called ‘repository’. Repository basically contains a collection of software. Ubuntu’s official repository has the softwares that are approved by Ubuntu. Canonical partner repository contains the softwares from partnered vendors.

In the same way, PPA enables a developer to create its own APT repository. When an end user (i.e you) adds this repository to the system (sources.list is modified with this entry), software provided by the developer in his/her repository becomes available for the user.

Now you may ask what’s the need of PPA when we already have the official Ubuntu repository?

The answer is that not all software automatically get added to Ubuntu’s official repository. Only the trusted software make it to that list. Imagine that you developed a cool Linux application and you want to provide regular updates to your users but it will take months before it could be added to Ubuntu’s repository (if it could). PPA comes handy in those cases.

Apart from that, Ubuntu’s official repository often doesn’t include the latest version of a software. This is done to secure the stability of the Ubuntu system. A brand new software version might have a regression that could impact the system. This is why it takes some time before a new version makes it to the official repository, sometimes it takes months.

But what if you do not want to wait till the latest version comes to the Ubuntu’s official repository? This is where PPA saves your day. By using PPA, you get the newer version.

Typically PPA are used in three commands. First to add the PPA repository to the sources list. Second to update the cache of software list so that your system could be aware of the new available software. And third to install the software from the PPA.

I’ll show you an example by using Numix theme PPA:

sudo add-apt-repository ppa:numix/ppa
sudo apt-get update
sudo apt-get install numix-gtk-theme numix-icon-theme-circle

In the above example, we added a PPA provided Numix project. And after updating the software information, we add two programs available in Numix PPA.

If you want a GUI application, you can use Y-PPA application. It lets you search for PPA, add and remove software in a better way.

Tip: Security of PPA has often debated. My advice is that you should add PPA from a trusted source, preferably from the official sources.

4.2 REMOVE APPLICATIONS INSTALLED USING PPA

I have discussed removing PPA in Ubuntu in detail earlier. You should refer to that article to get more insights about handling PPA removal.

To quickly discuss it here, you can use the following two commands.

sudo apt-get remove numix-gtk-theme numix-icon-theme-circle
sudo add-apt-repository --remove ppa:numix/ppa

First command removes the software installed via the PPA. Second command removes the PPA from sources.list.

5.1 INSTALLING SOFTWARE USING SOURCE CODE IN UBUNTU LINUX [NOT RECOMMENDED]

Installing a software using the source code is not something I would recommend to you. It’s tedious, troublesome and not very convenient. You’ll have to fight your way through dependencies and what not. You’ll have to keep the source code files else you won’t be able to uninstall it later.

But building from source code is still liked by a few, even if they are not developing software of their own. To tell you the truth, last I used source code extensively was 5 years ago when I was an intern and I had to develop a software in Ubuntu. I have preferred the other ways to install applications in Ubuntu since then. For normal desktop Linux user, installing from source code should be best avoided.

I’ll be short in this section and just list out the steps to install a software from source code:

  • Download the source code of the program you want to install.
  • Extract the downloaded file.
  • Go to extracted directory and look for a README or INSTALL file. A well-developed software may include such a file to provide installation and/or removal instructions.
  • Look for a file called configure. If it’s present, run the file using the command: ./configure This will check if your system has all the required softwares (called ‘dependencies’ in software term) to install the program. Note that not all software include configure file which is, in my opinion, bad development practice.
  • If configure notifies you of missing dependencies, install them.
  • Once you have everything, use the command make to compile the program.
  • Once the program is compiled, run the command sudo make install to install the software.

Do note that some softwares provide you with an install script and just running that files will install the software for you. But you won’t be that lucky most of the time.

Also note that the program you installed using this way won’t be updated automatically like programs installed from Ubuntu’s repository or PPAs or .deb.

I recommend reading this detailed article on using the source code in Ubuntu if you insist on using source code.

5.2 REMOVING SOFTWARE INSTALLED USING SOURCE CODE [NOT RECOMMENDED]

If you thought installing software from source code was difficult, think again. Removing the software installed using source code could be a bigger pain.

  • First, you should not delete the source code you used to install the program.
  • Second, you should make sure at the installation time that there is a way to uninstall the program. A badly configured program might not provide a way to uninstall the program and then you’ll have to manually remove all the files installed by the software.

Normally, you should be able to uninstall the program by going to its extracted directory and using this command:

sudo make uninstall

But this is not a guarantee that you’ll get this uninstall all the time.

You see, there are lots of ifs and buts attached with source code and not that many advantages. This is the reason why I do not recommend using the source code to install the software in Ubuntu.

FEW MORE WAYS TO INSTALL APPLICATIONS IN UBUNTU

There are a few more (not so popular) ways you can install software in Ubuntu. Since this article is already way too long, I won’t cover them. I am just going to list them here:

Linux commands

sudo!! : Forgot to run a command with sudo? You need not re-write the whole command, just type “sudo!!” and the last command will run with sudo.

2. Python -m SimpleHTTPServer : Creates a simple web page for the current working directory over port 8000.

3. mtr : A command which is a combination of ‘ping’ and ‘traceroute’ command.

4. Ctrl+x+e : This key combination fires up, an editor in the terminal, instantaneously.

5. nl : Outputs the content of text file with lines Numbered.

6. shuf : Randomly selects line/file/folder from a file/folder.

7. ss : Outputs Socket Statistics.

8. Last: Want to know history of last logged in users? This command comes to rescue here.

9. curl ifconfig.me : Shows machine’s external IP Address.

10. tree : Prints files and folders in tree like fashion, recursively.

11. Pstree : Prints running processes with child processes, recursively.

13. stat : Shows the status information of a file as well as of a file system.

15. Pv : outputs simulating text, similar to hollywood movies.

16. Mount | column -t : Lists mounted file system, in nice formatting with specification.

17. Ctrl + l: clear shell prompt, instantaneously.

18. curl -u gmail_id –silent “https://mail.google.com/mail/feed/atom” | perl -ne ‘print “t” if //; print “$2n” if /(.*)/;’. This simple scripts, opens up, unread mail of an user, in the terminal itself.

19. screen : Detach and Reattach, long running process from a session.

20. file : Outputs information, regarding types of file.

21. id : Print User and Group Id.
22. ^foo^bar : Run last command with modification, without the need of rewriting the whole command again.

24. at : Run a particular command, time based.

25. du -h –max-depth=1 Command : Outputs the size of all the files and folders within current folder, in human readable format.

26. expr : Solve simple mathematical calculations from the terminal.

27. look: Check for an English word, from the dictionary, in case of confusion, right from the shell.

28. yes : continues to print a sting, till interrupt instruction is given.

29. factor: Gives all the possible factors of a decimal number.

30. ping -i 60 -a IP_address : Pings the provided IP_address, and gives audible sound when host comes alive.

31. tac : Prints content of a file, in reverse order.

32. strace : A debugging tool.

33. disown -a && exit Command : Run a command in background, even after terminal session is closed.

34. getconf LONG_BIT Command : Output Machine Architecture, very clearly.

35. while sleep 1;do tput sc;tput cup 0 $(($(tput cols)-29));date;tput rc;done & : The script outputs date and time on the top right corner of shell/ terminal.

36. convert : converts the output of a command in picture, automatically.

37. watch -t -n1 “date +%T|figlet” : Show animated digital clock at the prompt.

38. host and dig : DNS lookup utility.

39. dstat : Generates statistics regarding system resource.

40. bind -p : Shows all the shortcuts available in Bash.

41. Touch /forcefsck : Force file-system check on next boot.

42. lsb_release : Prints distribution specification information.

43. nc -ZV localhost port_number : Check if a specific port is open or not.

44. curl ipinfo.io : Outputs Geographical Information, regarding an ip_address.

45. find .-user xyz : Lists all file owned by user ‘xyz’

46. apt-get build-dep package_name: Build all the dependency, automatically while installing any specific package.

47. lsof -iTCP:80 -sTCP:LISTEN. The script, outputs all the service/process using port 80.

48. find -size +100M : This command combination, Lists all the files/folders the size of which is 100M or more.

49. pdftk : A nice way to concatenate a lot of pdf files, into one.

50. ps -LF -u user_name : Outputs Processes and Threads of a user.

51. Startx — :1 (This command creates another new X session).

top 6 files that eat up your space:

du -hsx * | sort -rh | head -6
  • Alt+Backspace: Deletes the previous word.
  • Alt+F: Skips ahead to the next space.
  • Alt+B: Skips back to the previous space.
  • Ctrl+U: Cuts all text up to the cursor.
  • Ctrl+K: Cuts all text after the cursor until end of line.
  • Ctrl+A: Moves the cursor to the start of line.
  • Ctrl+E: Moves the cursor to the end of line.