Radek Sprta (Posts about automatization)https://radeksprta.eu/categories/automatization.atom2024-01-07T02:28:56ZRadek SprtaNikolaDeploy Docker Container from Gitlab CIhttps://radeksprta.eu/posts/deploy-docker-container-from-gitlab-ci/2020-09-12T10:46:27+02:002020-09-12T10:46:27+02:00Radek Sprta<div><p>Containers are all the rage nowadays and for a good reason. They help in unifying development and production environments. They also provide application encapsulation and isolation, among other things. But to get the most out of them, you should build and deploy them automatically. This post will show you how to do it using Gitlab CI and docker-compose.</p>
<p><a href="https://radeksprta.eu/posts/deploy-docker-container-from-gitlab-ci/">Read more…</a> (2 min remaining to read)</p></div>Modify All Items in Ansible Listhttps://radeksprta.eu/posts/modify-all-items-in-ansible-list/2020-08-08T08:58:40+02:002020-08-08T08:58:40+02:00Radek Sprta<p>Ansible lets you easily interpolate list items within values (like <code>interpolated_{{ item }}_value</code>). However, sometimes you need a more powerful transformation. This is where the map filter comes to rescue again. You can use it to perform regular expression replace on each item in a list. As you can see, the syntax is relatively simple:</p>
<div class="code"><pre class="code literal-block"><span class="nb">map</span><span class="p">(</span><span class="s1">'regex_replace'</span><span class="p">,</span> <span class="n">REGEX_PATTERN</span><span class="p">,</span> <span class="n">OUTPUT</span><span class="p">)</span>
</pre></div>
<p>For a concrete example, let us say you want to extract network mask from a list of IP addresses (<code>192.168.0.100/24</code> for example). Assuming this list is stored in the <code>ip_addresses</code> variable, the regex replace would look like this:</p>
<div class="code"><pre class="code literal-block"><span class="p">{{</span> <span class="n">ip_addresses</span> <span class="o">|</span> <span class="nb">map</span><span class="p">(</span><span class="s1">'regex_replace'</span><span class="p">,</span> <span class="s1">'.*/([0-9]{1,2})'</span><span class="p">,</span> <span class="s1">'</span><span class="se">\\</span><span class="s1">1'</span><span class="p">)</span> <span class="o">|</span> <span class="nb">list</span> <span class="p">}}</span>
</pre></div>
<p>Of course, you can easily use it as a part of longer jinja2 pipelines. If you also want to learn how to <a href="https://radeksprta.eu/posts/loop-over-dictionary-attribute-in-ansible/">loop over dictionary attribute</a>, or see other Ansible tips, take a look <a href="https://radeksprta.eu/categories/ansible/">here</a>.</p>Loop Over Dictionary Attribute in Ansiblehttps://radeksprta.eu/posts/loop-over-dictionary-attribute-in-ansible/2020-07-04T10:13:50+02:002020-07-04T10:13:50+02:00Radek Sprta<p>Working with variables can sometimes get tricky in Ansible. Say, you have a dictionary where you want loop over a certain attribute, not all values. For example, when your variables are declared like this:</p>
<div class="code"><pre class="code literal-block"><span class="nt">interfaces</span><span class="p">:</span>
<span class="w"> </span><span class="nt">eth0</span><span class="p">:</span>
<span class="w"> </span><span class="nt">ip</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">10.0.0.10</span>
<span class="w"> </span><span class="nt">mask</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">16</span>
<span class="w"> </span><span class="nt">eth1</span><span class="p">:</span>
<span class="w"> </span><span class="nt">ip</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">192.168.1.100</span>
<span class="w"> </span><span class="nt">mask</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">24</span>
</pre></div>
<p>How do you just loop over all the IP addresses? This is where the map filter comes in:</p>
<div class="code"><pre class="code literal-block"><span class="nt">vars</span><span class="p">:</span>
<span class="w"> </span><span class="nt">ip_addresses</span><span class="p">:</span><span class="w"> </span><span class="s">"{{</span><span class="nv"> </span><span class="s">interfaces.values()</span><span class="nv"> </span><span class="s">|</span><span class="nv"> </span><span class="s">map(attribute='container')</span><span class="nv"> </span><span class="s">|</span><span class="nv"> </span><span class="s">list</span><span class="nv"> </span><span class="s">}}"</span>
</pre></div>
<p>The snippet takes all values from the <code>interfaces</code> dictionary (<code>eth0</code>, <code>eth1</code>) and then extracts the <code>ip</code> attribute from them. Finally, it casts the result into list.</p>
<p><a href="https://docs.ansible.com/ansible/latest/user_guide/playbooks_filters.html">Ansible filters</a> are a very powerful tool, so I would recommend you take some time to read it thoroughly. If you are looking for more Ansible tips, such as <a href="https://radeksprta.eu/posts/make-ansible-playbook-distribution-agnostic">how to make a playbook distribution agnostic</a> or <a href="https://radeksprta.eu/posts/automatically-setup-computer-ansible-playbook">setup your laptop with Ansible</a>, look <a href="https://radeksprta.eu/categories/ansible/">here</a>.</p>One-liner to Download Latest PHPMyAdminhttps://radeksprta.eu/posts/one-liner-to-download-latest-phpmyadmin/2020-07-01T16:49:55+02:002020-07-01T16:49:55+02:00Radek Sprta<p>Even though the PHPMyAdmin's heyday is long gone by, it still remains quite popular. However,
unless you are running it as a Docker container or similar, it is difficult to maintain updated.
The reason being, that the creators do not offer any direct link to get the latest version. And
the version in system repositories is usually few releases behind.</p>
<p>Luckily, their website design has remained fairly stale, so you can scrape the download link
from there. The bash one-liner below (split into three lines for readability :)) will achieve
just that:</p>
<div class="code"><pre class="code literal-block">curl<span class="w"> </span>https://www.phpmyadmin.net/files/<span class="w"> </span><span class="m">2</span>>/dev/null<span class="w"> </span><span class="se">\</span>
<span class="w"> </span><span class="p">|</span><span class="w"> </span>grep<span class="w"> </span>-oP<span class="w"> </span><span class="s1">'(?<=href=")https://files.phpmyadmin.net/phpMyAdmin[^"]*(?=")'</span><span class="w"> </span><span class="se">\</span>
<span class="w"> </span><span class="p">|</span><span class="w"> </span>head<span class="w"> </span>-n<span class="w"> </span><span class="m">1</span>
</pre></div>
<p>It uses the nifty lookbehind feature (<code>(?<=)</code>) of the Perl-like expression matching. But that
is specific to GNU Grep, so you might need to install it, if you are on BSD or MacOS.</p>Automatic ClamAV Scans With Email Notificationshttps://radeksprta.eu/posts/automatic-clamav-scans-with-email-notifications/2018-09-18T09:42:52+02:002018-09-18T09:42:52+02:00Radek Sprta<div><p>You might consider antivirus software unneccessary on a Linux box. But if you
are running a public facing server, it might be a good idea to run it
regardless. Why is that? Even though malware probably can't affect your server
much, it can still use it to infect users. To prevent that, we will learn how
to install and set up automatic <a href="https://www.clamav.net/">ClamAV</a> scans with
email notifications. In the examples below, I am using Debian 9, but the
commands should be the same for any Linux distribution.</p>
<h3>Install and configure ClamAV</h3>
<p>You can easily install the antivirus by running the following command:</p>
<div class="code"><pre class="code literal-block">$ apt install clamav clamscan
</pre></div>
<p>ClamAV is pretty well
configured out of the box. Usually, I only change the frequency of virus
signature database updates. You can do that in <code>/etc/clamav/freshclam.conf</code> by
changing the <code>Checks</code> line. Since I run the scan once a day, I lower the
frequency of updates to that as well.</p>
<div class="code"><pre class="code literal-block">Checks 1
</pre></div>
<p><a href="https://radeksprta.eu/posts/automatic-clamav-scans-with-email-notifications/">Read more…</a> (1 min remaining to read)</p></div>Make Ansible Playbook Distribution Agnostichttps://radeksprta.eu/posts/make-ansible-playbook-distribution-agnostic/2017-07-25T18:53:27+02:002017-07-25T18:53:27+02:00Radek Sprta<div><p>I've been thinking about switching to a rolling distribution on my working
machine for quite some time. Because the packages in Ubuntu repositories are
just too outdated for my needs and I want to save myself the headache of PPAs.
And when <a href="https://insights.ubuntu.com/2017/04/05/growing-ubuntu-for-cloud-and-iot-rather-than-phone-and-convergence/">Ubuntu announced dropping Unity in favor of
Gnome</a>,
my mind was made up. I would use a
rolling distro on my main computer and Ubuntu 16.04 elsewhere.</p>
<h3>Unify package installation</h3>
<p>In the end, I decided to go for OpenSuse. Since I use an <a href="https://radeksprta.eu/posts/automatically-setup-computer-ansible-playbook/">Ansible playbook to
setup my machines</a>,
it would take no time to have it up an running. Or so I
thought. However, there was a little caveat. As my playbook used the apt
module to install software, it would obviously not run on OpenSuse. Luckily,
since the version 2.0, Ansible provides a platform-agnostic package manager
module. It is called simply <code>package</code>. As a result, installing packages across
different distributions is as easy as running:</p>
<div class="code"><pre class="code literal-block"><span class="nn">---</span>
<span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">Install packages</span>
<span class="w"> </span><span class="nt">become</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">yes</span>
<span class="w"> </span><span class="nt">package</span><span class="p">:</span>
<span class="w"> </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">git</span>
<span class="w"> </span><span class="nt">state</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">present</span>
</pre></div>
<p>Or is it?
While it may work in the case of Git, not all packages have a same name in
different Linux distributions. So you have to set up variables for those:
First of all, defaults in <code>vars/default.yml</code>:</p>
<div class="code"><pre class="code literal-block"><span class="nt">apache_package</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">apache2</span>
</pre></div>
<p>Then, for Debian family <code>vars/Debian.yml</code></p>
<div class="code"><pre class="code literal-block"><span class="nn">---</span>
<span class="nt">apache_package</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">apache2</span>
</pre></div>
<p>And for Redhat family <code>vars/Redhat.yml</code></p>
<div class="code"><pre class="code literal-block"><span class="nn">---</span>
<span class="nt">apache_package</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">httpd</span>
</pre></div>
<p>Afterwards, just include
them in your tasks:</p>
<div class="code"><pre class="code literal-block"><span class="w"> </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">include_vars</span><span class="p">:</span><span class="w"> </span><span class="s">"{{</span><span class="nv"> </span><span class="s">item</span><span class="nv"> </span><span class="s">}}"</span>
<span class="w"> </span><span class="nt">with_first_found</span><span class="p">:</span>
<span class="w"> </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="s">"{{</span><span class="nv"> </span><span class="s">ansible_os_family</span><span class="nv"> </span><span class="s">}}.yml"</span>
<span class="w"> </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="s">"default.yml"</span>
<span class="w"> </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">Install Apache</span>
<span class="w"> </span><span class="nt">become</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">yes</span>
<span class="w"> </span><span class="nt">package: name</span><span class="p">:</span><span class="w"> </span><span class="s">"{{</span><span class="nv"> </span><span class="s">apache_package</span><span class="nv"> </span><span class="s">}}"</span>
<span class="w"> </span><span class="nt">state</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">present</span>
</pre></div>
<p><a href="https://radeksprta.eu/posts/make-ansible-playbook-distribution-agnostic/">Read more…</a> (1 min remaining to read)</p></div>Automatically Setup Your Computer with Ansible Playbookhttps://radeksprta.eu/posts/automatically-setup-computer-ansible-playbook/2017-01-08T22:50:54+01:002017-01-08T22:50:54+01:00Radek Sprta<div><p>Setting up a computer can be a major annoyance. Being 'the tech guy', I still
have vivid nightmares of spending every other weekend reinstalling friends'
and family members' Windows boxes that have come to a crawl. Now, most of us
know enough to backup the important data and that makes the process easier.
(If you don't have backups, go setup
<a href="https://radeksprta.eu/installing-nextcloud-on-raspberry-pi/">Nextcloud</a>,
<a href="https://syncthing.net/">Syncthing</a>, or some other automatic system.
Seriously, it's gonna come back to bite you sooner or later.) But there is
still the matter of installing and configuring all your software. Some people
have tackled that by writing install scripts. However, there is a better way -
Ansible playbook. Let me introduce Ansible. Ansible is a software for
computer provisioning. In computing, provisioning means automated installation
and configuration of software and services. Provisioning tools, such as
Ansible, Chef and Puppet, help you cut down the setup time from hours to mere
minutes. Their main advantage over custom scripts is that they are idempotent.
That means they only makes the necessary changes. So, if a task fails (for
example because ou forgot to connect your Ethernet cable, oops), you can just
run it again without any worries. The reason I chose to use Ansible, is that
the tasks are easily configurable using yaml files. It is also written in
Python, which makes it easy to write your own module for it, in case of need.
So what is Ansible playbook? It is the collection of yaml files, describing
the tasks to be executed. You can find an introduction to its syntax in
<a href="https://docs.ansible.com/ansible/latest/user_guide/playbooks_intro.html#about-playbooks">Ansible documentation</a>.
I will demonstrate how you can setup your computers using an Ansible playbook,
using mine as an example. You can find it at <a href="https://gitlab.com/radek-sprta/ansible-personal">my Gitlab
page</a>.</p>
<p><a href="https://radeksprta.eu/posts/automatically-setup-computer-ansible-playbook/">Read more…</a> (3 min remaining to read)</p></div>Autostart Linux Applications in One Commandhttps://radeksprta.eu/posts/autostart-linux-applications-one-command/2016-11-18T23:03:23+01:002016-11-18T23:03:23+01:00Radek Sprta<p>Usually, when I am done working on the computer, I just suspend it. Thanks to
that, I can easily pick up where I left off. But sometimes, I have to swallow
the pill and turn it off (for example to preserve charge while traveling). In
such cases, you can autostart Linux apps to quickly setup your work
environment and get back to being productive. Fortunately, you can easily set
this up in desktop environments adhering to
<a href="https://www.freedesktop.org/wiki/">Freedesktop</a> standards. These include
Gnome and KDE, among others. You just need to copy the .desktop files of your
favorite programs from <code>/usr/share/applications/</code> to the autostart directory.
For instance your browser, password manager or a <a href="https://github.com/Guake/guake">roll-down
terminal</a>. If you only want it to start for
yourself, copy it to <code>$HOME/.config/autostart</code>. However, for system-wide
configuration, the directory is <code>/etc/xdg/autostart</code>. As a result, you don't
have to waste time starting up the same 5 applications every time you reboot.
And you can set everything up using just command line. So you can even use it
is part of your computer installation script. If you are looking for other
ways to automize tedious tasks, check out this <a href="https://radeksprta.eu/automatically-setup-computer-ansible-playbook/">guide to automatically
configure your computer with Ansible</a>.</p>