tag:blogger.com,1999:blog-80320228112351827592024-03-16T23:52:40.426-07:00Linux CommandoInitially a Linux command-line interface blog, it has evolved to cover increasingly more GUI app topics. Instead of just giving you information like some man page, I illustrate their usage in real-life scenarios.Peter Leunghttp://www.blogger.com/profile/05589860210899238688noreply@blogger.comBlogger197125tag:blogger.com,1999:blog-8032022811235182759.post-77466374713677917402020-07-13T03:00:00.000-07:002020-07-13T03:00:06.233-07:00Useful command-line date utilities<div class="separator" style="clear: both; text-align: center;"><a href="https://2.bp.blogspot.com/-cHpdN5gzxn0/Xv9zS5XoeKI/AAAAAAAADDM/OlROmvpTVgIG1VHnXsAurTz3FLy5CHwxACLcBGAsYHQ/s1600/calendar-3109374_640.jpg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://2.bp.blogspot.com/-cHpdN5gzxn0/Xv9zS5XoeKI/AAAAAAAADDM/OlROmvpTVgIG1VHnXsAurTz3FLy5CHwxACLcBGAsYHQ/s320/calendar-3109374_640.jpg" width="320" height="200" data-original-width="640" data-original-height="399" /></a></div>
<p>My earlier post, <a href="https://linuxcommando.blogspot.com/2009/11/fun-with-date-arithmetic.html">Fun with Date Arithmetic</a> shows how to use the <em>date</em> command to compute a future or past date that is a certain number of days ahead or in the past. This post expands on how you can manipulate dates on the Linux command-line interface using the <em>cal</em> and <a href="http://www.fresse.org/dateutils/"><em>dateutils</em></a> programs. These commands can give you results that GUI calendars can't.</p>
<h2>cal</h2>
<p><p>While the <em>date</em> command works with the basic unit of days, say display the day that is 3 days from today, the <em>cal</em> command manipulates months, say display the month that is 2 months away.
<p>Running <em>cal</em> without any argument displays the calendar for the current month.
<pre><code>$ cal
July 2020
Su Mo Tu We Th Fr Sa
1 2 3 4
5 6 7 8 9 10 11
12 13 14 15 16 17 18
19 20 21 22 23 24 25
26 27 28 29 30 31
</code></pre>
<p>While GUI calendar programs are well capable of displaying single month calendars, they are no match to the <em>cal</em> command for simultaneously displaying multiple months. <p> To display the current and the following say 2 months, use the <em>-A2</em> argument:
<pre><code>$ cal -A2
2020
July August September
Su Mo Tu We Th Fr Sa Su Mo Tu We Th Fr Sa Su Mo Tu We Th Fr Sa
1 2 3 4 1 1 2 3 4 5
5 6 7 8 9 10 11 2 3 4 5 6 7 8 6 7 8 9 10 11 12
12 13 14 15 16 17 18 9 10 11 12 13 14 15 13 14 15 16 17 18 19
19 20 21 22 23 24 25 16 17 18 19 20 21 22 20 21 22 23 24 25 26
26 27 28 29 30 31 23 24 25 26 27 28 29 27 28 29 30
30 31
</code></pre>
<p> To display past months is equally easy with the <em>-B</em> argument. In fact, you can combine <em>-B</em> and <em>-A</em> to, for example, display the previous and the next month as follows:
<pre><code>$ cal -A1 -B1
June 2020 July 2020 August 2020
Su Mo Tu We Th Fr Sa Su Mo Tu We Th Fr Sa Su Mo Tu We Th Fr Sa
1 2 3 4 5 6 1 2 3 4 1
7 8 9 10 11 12 13 5 6 7 8 9 10 11 2 3 4 5 6 7 8
14 15 16 17 18 19 20 12 13 14 15 16 17 18 9 10 11 12 13 14 15
21 22 23 24 25 26 27 19 20 21 22 23 24 25 16 17 18 19 20 21 22
28 29 30 26 27 28 29 30 31 23 24 25 26 27 28 29
30 31
</code></pre>
<p> <em>cal</em> has a special shortcut for displaying the above combination, namely, the current month with the immediate before and after month:
<pre><code>$ cal -3
</code></pre></p>
<h2>dateutils</h2>
<div class="separator" style="clear: both; text-align: center;"><a href="https://2.bp.blogspot.com/-ducgcYeYu30/XwdoUlFQSCI/AAAAAAAADDs/GWoDZEeqETwGdYPct5mX5Xq-GkudLD3MgCLcBGAsYHQ/s1600/abacus-33275_640.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://2.bp.blogspot.com/-ducgcYeYu30/XwdoUlFQSCI/AAAAAAAADDs/GWoDZEeqETwGdYPct5mX5Xq-GkudLD3MgCLcBGAsYHQ/s200/abacus-33275_640.png" width="200" height="142" data-original-width="640" data-original-height="455" /></a></div>
<p><p> While <em>date</em> is a nifty program to do <a href="https://linuxcommando.blogspot.com/2009/11/fun-with-date-arithmetic.html">date arithemetic</a>, <em>dateutils</em> is a more versatile collection of tools for date manipulation.
<p> To install the <em>dateutils</em> program on Debian,
<pre><code>$ sudo apt install dateutils
</code></pre></p>
<h3>dateutils.dadd</h3>
<p><p> Use the <em>dateutils.dadd</em> sub-command to do date arithmetic. The program takes 2 input: a date and a duration either before or after the said date, specified in years, months, weeks, or days. For instance, to output the date that is 1 week and 2 days in the future from today, say 2020-07-10:
<pre><code>$ dateutils.dadd today +1w2d
2020-07-19
</code></pre>
<p> You can replace the special keyword <em>today</em> with a specific date. For instance, to compute the date that is 1 year, 2 months and 3 days in the past from 2020-07-10:
<pre><code>$ dateutils.dadd 2020-07-10 -1y2m3d
2019-05-07
</code></pre></p>
<h3>dateutils.ddiff</h3>
<p><p>Given 2 dates, <em>dateutils.ddiff</em> computes the duration between them. The default output unit is the number of days. You can customize the output units by specifying a format using the <em>-f</em> argument.
<pre><code>$ dateutils.ddiff 2020-07-02 2020-08-21 -f "%y years %m months %w weeks %d days"
0 years 1 months 2 weeks 5 days
</code></pre></p>
<h2>Related posts:</h2>
<p>. <a href="https://linuxcommando.blogspot.com/2009/11/fun-with-date-arithmetic.html">Fun with Date Arithmetic</a></p>
<div class="blogger-post-footer"><script type="text/javascript"><!--
google_ad_client = "pub-1640070864477148";
//728x90, created 11/11/07
google_ad_slot = "6009857287";
google_ad_width = 728;
google_ad_height = 90;
//--></script>
<script type="text/javascript"
src="http://pagead2.googlesyndication.com/pagead/show_ads.js">
</script></div>Peter Leunghttp://www.blogger.com/profile/05589860210899238688noreply@blogger.com4tag:blogger.com,1999:blog-8032022811235182759.post-3369617829498101942020-07-06T03:00:00.000-07:002020-07-06T03:00:14.370-07:00Use Certbot to renew Let's Encrypt TLS certificates<div class="separator" style="clear: both; text-align: center;">
<a href="https://3.bp.blogspot.com/-Q3ZwDEUkxm4/XvvZMKd7gPI/AAAAAAAADB8/TKzQrLjPiOIQ3sh_-p-mX_qt6vrk7rM6wCLcBGAsYHQ/s1600/https-3344700_640%2B%25281%2529.jpg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="256" data-original-width="640" height="80" src="https://3.bp.blogspot.com/-Q3ZwDEUkxm4/XvvZMKd7gPI/AAAAAAAADB8/TKzQrLjPiOIQ3sh_-p-mX_qt6vrk7rM6wCLcBGAsYHQ/s200/https-3344700_640%2B%25281%2529.jpg" width="200" /></a></div>
<p>You created a new website or perhaps even configured a SMTP mail server. You patted yourself on the back because you had not forgotten about securing the web and mail services. Specifically, you set up a free TLS certificate from <a href="https://www.blogger.com/blogger.g?blogID=8032022811235182759">Let's Encrypt</a>. So, it is time to put your feet up and admire the good work you had done, right?
<p>Not yet. The TLS certificate from Let's Encrypt would expire every 90 days, and is renewable only after 60 days. Doesn't this scream for automation?
<p>Assuming that you have shell access to the host server, this blog post explains how to use <em>Certbot</em> to automate the renewal of Let's Encrypt certificates, and points out some gotchas to avoid.
<h2>
Certbot</h2>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://2.bp.blogspot.com/-d-hXQtI1eI8/Xvy5oOEt6_I/AAAAAAAADCY/YDn9W1D7dSgOzXDUN-xMx1ULKPYkbHr_ACLcBGAsYHQ/s1600/17889013.png" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" data-original-height="265" data-original-width="265" height="200" src="https://2.bp.blogspot.com/-d-hXQtI1eI8/Xvy5oOEt6_I/AAAAAAAADCY/YDn9W1D7dSgOzXDUN-xMx1ULKPYkbHr_ACLcBGAsYHQ/s200/17889013.png" width="200" /></a></div>
<p>The recommended way to deploy Let's Encrypt certificates on a Linux system is to use the <a href="https://certbot.eff.org/">certbot</a> tool. This tutorial assumes that you have successfully used <em>certbot</em> to obtain and install a Let's Encrypt certificate.
<p>The Let's Encrypt ecosystem with <em>certbot</em> is designed with automation in mind. When you install <em>certbot</em> <a href="https://certbot.eff.org/docs/using.html?highlight=renew#automated-renewals">on various Linux distributions</a> such as Debian, Ubuntu, Fedora, CentOS, etc, the mechanism for certificate renewal is already put in place. What you need to do is make sure that the timer for <em>certbot</em> is enabled.
<pre><code>$ systemctl status certbot.timer
● certbot.timer - Run certbot twice daily
Loaded: loaded (/lib/systemd/system/certbot.timer; enabled; vendor preset: enabled)
Active: active (waiting) since Sat 2020-06-27 10:20:51 PDT; 3 days ago
Trigger: Wed 2020-07-01 12:14:52 PDT; 2h 1min left
</code></pre>
<p>To enable the timer,
<pre><code>$ sudo systemctl enable certbot.timer
</code></pre>
<p><em>Certbot</em> is scheduled to automatically run twice daily to check if certificate renewal is needed. A certificate is only renewed, however, if expiry is impending—within 30 days before expiration.
<p>For your peace of mind, you can verify the current status of your TLS certificates using the web tools <a href="https://www.ssllabs.com/ssltest/">SSL Test</a> and <a href="https://crt.sh/">crt.sh</a>.
<h2>
Gotcha # 1: Refreshing certificates</h2>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-fdiTCbAgklc/XvzZfwgBfJI/AAAAAAAADCk/En6srRYqDd0NFK59hkcMQh1K0svk6hbhwCLcBGAsYHQ/s1600/happy-playful-young-man.jpg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="1068" data-original-width="1600" height="134" src="https://1.bp.blogspot.com/-fdiTCbAgklc/XvzZfwgBfJI/AAAAAAAADCk/En6srRYqDd0NFK59hkcMQh1K0svk6hbhwCLcBGAsYHQ/s200/happy-playful-young-man.jpg" width="200" /></a></div>
<p>Renewing a certificate before it expires is only half the battle. The other half is to get the web server (and email server if applicable) to use the new certificate. The obsolete version is still in use until the webserver (and email server) is reloaded.
<p><em>Certbot</em> provides a hook interface to run scripts before or after a certificate is renewed. I used the <em>Deploy</em> hooks and the <em>Pre-hooks</em> to automate the reloading of web and email server programs.
<h3>
Deploy hooks</h3>
<p>A <em>Deploy</em> hook is run <strong>after</strong> the successful renewal of a certificate. Deploy hooks are placed in the <em>/etc/letsencrypt/renewal-hooks/deploy</em> directory.
<p>If there is more than 1 script in the directory, the scripts are executed in alphabetical order based on their filenames.
<p>I created 2 Deploy scripts, <em>01-reload-nginx.sh</em> and <em>02-reload-postfix.sh</em> to reload <em>NGINX</em> and <em>postfix</em> respectively. Both scripts should be executable(file permissions set to 770).
<p>A typical script to reload <em>NGINX</em> is as follows.
<pre><code>$ sudo cat /etc/letsencrypt/renewal-hooks/deploy/01-reload-nginx.sh
#! /bin/sh
set -e
/etc/init.d/nginx configtest
/etc/init.d/nginx reload
</code></pre>
<p>To switch to the new certificate for <i>postfix</i>, both <em>postfix</em> and <em>dovecot</em> need to be reloaded.
<pre><code>$ sudo cat /etc/letsencrypt/renewal-hooks/deploy/02-reload-postfix.sh
#! /bin/sh
set -e
/etc/init.d/postfix reload
/etc/init.d/dovecot reload
</code></pre>
<h3>
Pre-hooks</h3>
<p><em>Pre-hooks</em> are scripts to be run when a certificate is due for renewal, i.e., within 30 days prior to expiration, and <strong>before</strong> the renewal is actually performed. Those scripts are placed in the <em>/etc/letsencrypt/renewal-hooks/pre/</em> directory.
<p>I created a <em>Pre-hook</em> script named <em>01-notify-renewal.sh</em> to email me when a certificate is due for renewal.
<pre><code>$ sudo cat /etc/letsencrypt/renewal-hooks/pre/01-notify-renewal.sh
#! /bin/sh
set -e
echo 'Message' | /usr/bin/mail <a href="mailto:yourEmail@example.com">yourEmail@example.com</a> -s 'Subject'
</code></pre>
<h2>
Manual execution</h2>
<p>After auto-renewal is set up, there is little need to manually renew a certificate. But, you have the option to do so. To manually renew:
<pre><code>$ sudo certbot -q renew
</code></pre>
<p>The <em>-q</em> option suppresses all output except errors.
<p>Recall that a certificate will only be renewed if it is within 30 days to expiry. You can override this restriction by specifying the <em>--force-renewal</em> option. Use it with caution however(see gotcha # 2).
<pre><code>$ sudo certbot -q --force-renewal renew
</code></pre>
<h2>
Gotcha # 2: Rate limit</h2>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://2.bp.blogspot.com/-tt6wNpUx2A8/Xv0N6edGlcI/AAAAAAAADC8/v6j0OOmVJzAwc3ylzUu_qEi-X_PKLT8vQCLcBGAsYHQ/s1600/limits-2235709_640.png" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" data-original-height="333" data-original-width="640" height="104" src="https://2.bp.blogspot.com/-tt6wNpUx2A8/Xv0N6edGlcI/AAAAAAAADC8/v6j0OOmVJzAwc3ylzUu_qEi-X_PKLT8vQCLcBGAsYHQ/s200/limits-2235709_640.png" width="200" /></a></div>
<p>Renewal is subject to a Duplicate Certificate limit of 5 per week. Please read the <a href="https://letsencrypt.org/docs/rate-limits/">rate limits documentation</a> to be acquainted with what counts toward this limit.
<p>If the rate limit for a certificate is exceeded, renewal is temporarily suspended until the rate limit resets—on a sliding basis—after a week. For instance, if you renew a certificate 3 times on Monday and twice more on Friday, renewal is suspended until the following Monday.
<div class="blogger-post-footer"><script type="text/javascript"><!--
google_ad_client = "pub-1640070864477148";
//728x90, created 11/11/07
google_ad_slot = "6009857287";
google_ad_width = 728;
google_ad_height = 90;
//--></script>
<script type="text/javascript"
src="http://pagead2.googlesyndication.com/pagead/show_ads.js">
</script></div>Peter Leunghttp://www.blogger.com/profile/05589860210899238688noreply@blogger.com0tag:blogger.com,1999:blog-8032022811235182759.post-41229759742943575762020-07-02T03:00:00.000-07:002020-07-02T03:00:05.411-07:00How to generate and read QR code on Linux<div class="separator" style="clear: both; text-align: center;"><a href="https://2.bp.blogspot.com/-Zko4S1LoYX0/XvUqgfZvzuI/AAAAAAAADBQ/YODelTahLkYNO0q6gIxZd5o6K7kE5cg7gCLcBGAsYHQ/s1600/qr-code-3970681_640.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://2.bp.blogspot.com/-Zko4S1LoYX0/XvUqgfZvzuI/AAAAAAAADBQ/YODelTahLkYNO0q6gIxZd5o6K7kE5cg7gCLcBGAsYHQ/s320/qr-code-3970681_640.jpg" width="320" height="182" data-original-width="640" data-original-height="363" /></a></div><p><p><a href="https://www.qrcode.com/en/index.html">QR code</a>, short for Quick Response code, was initially created to improve on bar codes used in inventory management. Nowadays, QR codes are ubiquitous, on posters, billboards, web pages, etc. This post will illustrate how to generate and read QR codes using the Linux command line interface (CLI).<br />
<p>The programs you will need to generate and read QR codes are <em>qrencode</em> and <em>zbarimg</em> respectively. (If you want to work with a GUI tool, there is QtQR.) To install the 2 programs on Debian:<br />
<pre><code>$ sudo apt install qrencode zbar-tools
</code></pre></p><h2>Background</h2><div class="separator" style="clear: both; text-align: center;"><a href="https://3.bp.blogspot.com/--Pkbum0bLOo/XvQT8quuG2I/AAAAAAAADAo/Nq4o9EmR6mgsS-QBF7dW01VbT08Zo4rlwCLcBGAsYHQ/s1600/webURL.png" imageanchor="1" style="clear: left; float: right; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://3.bp.blogspot.com/--Pkbum0bLOo/XvQT8quuG2I/AAAAAAAADAo/Nq4o9EmR6mgsS-QBF7dW01VbT08Zo4rlwCLcBGAsYHQ/s200/webURL.png" width="200" height="200" data-original-width="111" data-original-height="111" /></a></div><p><p>A QR code is a matrix of square dots (or 'modules' in QRspeak). QR codes have as many as 40 versions of increasing data capacity. Version 1's dimension is 21 × 21 modules, and each higher version adds 4 modules per side ending with version 40 with 177 × 177 modules.<br />
<p>The <a href="https://www.qrcode.com/en/about/version.html">exact maximum data capacity</a> of a version depends on several factors, including the type of characters stored, e.g., numeric vs alphanumeric, and the level of error correction desired. At Medium error correcting capability, version 1 can store up to 20 alphanumeric characters; version 40, 3,391.<br />
<p>Fortunately, as we'll see next, the <em>qrencode</em> utility specifies good defaults, and hides much of the gory details from you.</p><h2>QR code generation</h2><p><p>In its simplest form, <em>qrencode</em> takes the input string to be encoded and outputs the PNG graphic to a file. The following command encodes the URL for this website.<br />
<pre><code>$ qrencode -o webURL.png '<a href="https://linuxcommando.blogspot.com/">https://linuxcommando.blogspot.com/</a>'
</code></pre><p>You can specify different parameters to fine-tune the QR code. Use the <em>-l</em> parameter to change the error correction level from the default <em>L</em> for Lowest to <em>M</em> for Medium, <em>Q</em> for Quite High, or <em>H</em> for Highest. In addition, you can explicitly specify the version to use, the size of the module and the margin, etc. The following example generates a version two QR code for the same website at the Highest error correcting level.<br />
<pre><code>$ qrencode -o webURL.png -l H -v 2 '<a href="https://linuxcommando.blogspot.com/">https://linuxcommando.blogspot.com/</a>'
</code></pre><p>Besides the URL, marketers typically encode information such as phone numbers and email addresses.<br />
<pre><code>$ qrencode -o webPhone.png '(604)555-1234'
$ qrencode -o webEmail.png '<a href="mailto:spanish3rdlanguage@gmail.com">spanish3rdlanguage@gmail.com</a>'
</code></pre><p>Many QR code scanners will automatically open the associated app upon scanning a QR code of a special format, e.g., a browser for URLs, email client for email addresses, and phone app for telephone numbers.</p><h2>QR code scanning</h2><p><p>The Linux program <em>zbarimg</em> decodes the QR code stored in a file. To invoke, simply provide the input filename which contains the QR code.<br />
<pre><code>$ zbarimg webURL.png
QR-Code:<a href="https://linuxcommando.blogspot.com/">https://linuxcommando.blogspot.com/</a>
scanned 1 barcode symbols from 1 images in 0 seconds
</code></pre><p>If you specify the <em>-d</em> parameter, <em>zbarimg</em> will display the QR code in addition to the decoded information.<br />
<div class="separator" style="clear: both; text-align: center;"><a href="https://2.bp.blogspot.com/-75dkpIoCD-s/XvTt4Ke13pI/AAAAAAAADA0/_2pnAwe3vD0-vwDRwaBWDlPk7UveqqRSwCLcBGAsYHQ/s1600/code-4425886_640.jpg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://2.bp.blogspot.com/-75dkpIoCD-s/XvTt4Ke13pI/AAAAAAAADA0/_2pnAwe3vD0-vwDRwaBWDlPk7UveqqRSwCLcBGAsYHQ/s200/code-4425886_640.jpg" width="200" height="121" data-original-width="640" data-original-height="387" /></a></div><p>The default camera app of recent Android or iOS phones can also function as QR code scanner. To scan, run the camera app and point it towards the QR code.</p><div class="blogger-post-footer"><script type="text/javascript"><!--
google_ad_client = "pub-1640070864477148";
//728x90, created 11/11/07
google_ad_slot = "6009857287";
google_ad_width = 728;
google_ad_height = 90;
//--></script>
<script type="text/javascript"
src="http://pagead2.googlesyndication.com/pagead/show_ads.js">
</script></div>Peter Leunghttp://www.blogger.com/profile/05589860210899238688noreply@blogger.com1tag:blogger.com,1999:blog-8032022811235182759.post-8749137077402760132020-06-23T12:22:00.000-07:002020-06-23T12:22:43.596-07:00 Adding Google Analytics tracking code to a WordPress website<div class="separator" style="clear: both; text-align: center;"><a href="https://2.bp.blogspot.com/-X2qFln4pHb4/Xu0x9pDaa7I/AAAAAAAAC98/AYoSWcQIqlsGdrwWeNxA01pmV_r5O-6AgCLcBGAsYHQ/s1600/download%2B%25282%2529.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="143" data-original-width="351" height="130" src="https://2.bp.blogspot.com/-X2qFln4pHb4/Xu0x9pDaa7I/AAAAAAAAC98/AYoSWcQIqlsGdrwWeNxA01pmV_r5O-6AgCLcBGAsYHQ/s320/download%2B%25282%2529.png" width="320" /></a></div>My <a href="https://linuxcommando.blogspot.com/2020/06/how-to-migrate-single-site-wordpress-to.html">previous post</a> revealed how to piggyback a new WordPress website on an existing WordPress instance using what is known in WordPress-speak as multisite. In this post, I'll walk through how to embed the Google Analytics tracking code, aka the Global Site Tag, in the new website.<br />
<p>Website administrators want to know who, when, how, and what users do on their websites. Google Analytics can provide that information if the proper tracking code is found on the web pages. <br />
<p>There is more than 1 way to insert the tracking code, including using the Google Tag Manager or the WordPress plugin MonsterInsights. This post details a manual method of directly embedding the tracking code in a WordPress theme, assuming that you have already set up a <a href="https://analytics.google.com/">Google Analytics</a> account for the target website. In addition, I assume that you have created and activated a child theme for your website. If you are hosting multiple websites within the same WordPress instance, I assume you have a separate child theme for each site.<br />
<ol><li> <a href="https://analytics.google.com/">Login</a> to Google Analytics, and select the account corresponding to the target website.<br />
<div class="separator" style="clear: both; text-align: center;"><a href="https://3.bp.blogspot.com/-9s9iZNIi11U/Xu_AXvMtUsI/AAAAAAAAC_U/BDGYBI76GucP9LlmE3tGRj4evxobrEByACLcBGAsYHQ/s1600/Screenshot%2Bfrom%2B2020-06-21%2B13-15-56.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="817" data-original-width="559" height="320" src="https://3.bp.blogspot.com/-9s9iZNIi11U/Xu_AXvMtUsI/AAAAAAAAC_U/BDGYBI76GucP9LlmE3tGRj4evxobrEByACLcBGAsYHQ/s320/Screenshot%2Bfrom%2B2020-06-21%2B13-15-56.png" width="219" /></a></div></li>
<li> Click <i>Admin</i> on the LHS menu bar.<br />
</li>
<div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-OyxGOV8pNpw/XvI1divNdGI/AAAAAAAAC_4/kC4aH1SvaLQ97TC4C7rLabSdRkmTl_magCLcBGAsYHQ/s1600/Screenshot%2Bfrom%2B2020-06-23%2B09-58-59.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="760" data-original-width="1414" height="171" src="https://1.bp.blogspot.com/-OyxGOV8pNpw/XvI1divNdGI/AAAAAAAAC_4/kC4aH1SvaLQ97TC4C7rLabSdRkmTl_magCLcBGAsYHQ/s320/Screenshot%2Bfrom%2B2020-06-23%2B09-58-59.png" width="320" /></a></div><li> Click <i>Tracking Info</i> and then <i>Tracking Code</i> in the middle column.<br />
</li>
<div class="separator" style="clear: both; text-align: center;"></div><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-vEkSjuUu8CM/XvI7rrXFQxI/AAAAAAAADAI/E3gyH-EQmSQ-NCgTIrOYyb-6LnIY136UACLcBGAsYHQ/s1600/Screenshot%2Bfrom%2B2020-06-23%2B10-21-25.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="753" data-original-width="1331" height="181" src="https://1.bp.blogspot.com/-vEkSjuUu8CM/XvI7rrXFQxI/AAAAAAAADAI/E3gyH-EQmSQ-NCgTIrOYyb-6LnIY136UACLcBGAsYHQ/s320/Screenshot%2Bfrom%2B2020-06-23%2B10-21-25.png" width="320" /></a></div><li> Copy the <i>Global Site Tag</i> script to be pasted next in the WordPress theme.<br />
</li>
<li>SSH into web host, and copy the <i>header.php</i> file from the parent theme to the children theme.<br />
</li>
<pre><code>$ cp /var/www/example1.com/wp-content/themes/twentyseventeen/header.php /var/www/example1.com/wp-content/themes/twentyseventeen-child/header.php
</code></pre><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-1T07YuZA0ws/XvJVLf-pRBI/AAAAAAAADAU/tVzi74cU0D46nWD7rBcFojYuRtYG3VkEQCLcBGAsYHQ/s1600/Screenshot%2Bfrom%2B2020-06-23%2B12-14-53.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://1.bp.blogspot.com/-1T07YuZA0ws/XvJVLf-pRBI/AAAAAAAADAU/tVzi74cU0D46nWD7rBcFojYuRtYG3VkEQCLcBGAsYHQ/s320/Screenshot%2Bfrom%2B2020-06-23%2B12-14-53.png" width="320" height="104" data-original-width="879" data-original-height="287" /></a></div><li> Paste the tracking code script. <br />
The script should be inserted in the <i>header.php</i> file in your child theme directory, say <i>/var/www/example1.com/wp-content/themes/twentyseventeen-child, </i>just before the call to <i>wp_head()</i> at the end of the header specification.</li>
<div class="separator" style="clear: both; text-align: center;"><a href="https://3.bp.blogspot.com/-TI12Urg79Bo/Xu-x6PLFMpI/AAAAAAAAC-s/tWU_aCbeUCA_vmRuvs4o62C6L4-_qllBgCLcBGAsYHQ/s1600/Screenshot%2Bfrom%2B2020-06-21%2B12-14-00.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="375" data-original-width="919" height="131" src="https://3.bp.blogspot.com/-TI12Urg79Bo/Xu-x6PLFMpI/AAAAAAAAC-s/tWU_aCbeUCA_vmRuvs4o62C6L4-_qllBgCLcBGAsYHQ/s320/Screenshot%2Bfrom%2B2020-06-21%2B12-14-00.png" width="320" /></a></div><li> Navigate back to <i>Tracking Info/Tracking Code</i> on Google Analytics and click <i>Send Test Traffic</i>.<br />
A new session of your website pops up in the browser.<a href="https://3.bp.blogspot.com/-IRTaD3DV_d0/Xu-01ERcweI/AAAAAAAAC-8/VWH25jR2gjc9-z-6fRqvRcBR1vUxoNejwCLcBGAsYHQ/s1600/Screenshot%2Bfrom%2B2020-06-21%2B12-27-07.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em; text-align: center;"><img border="0" data-original-height="359" data-original-width="694" height="166" src="https://3.bp.blogspot.com/-IRTaD3DV_d0/Xu-01ERcweI/AAAAAAAAC-8/VWH25jR2gjc9-z-6fRqvRcBR1vUxoNejwCLcBGAsYHQ/s320/Screenshot%2Bfrom%2B2020-06-21%2B12-27-07.png" width="320" /></a></li>
<li> Navigate to <i>Reports</i> section on the LHS menu bar, click <i>Realtime</i>, then <i>Overview</i>.<br />
You should see the just opened session being counted in the number of <i>active users on site</i>.<br />
<div class="separator" style="clear: both; text-align: center;"><a href="https://3.bp.blogspot.com/-8JXmWjGr1Ng/Xu-8AUAg_UI/AAAAAAAAC_I/WXa4-rpnd1Q8vjLPFDmei8vgOdgWLjxqACLcBGAsYHQ/s1600/Screenshot%2Bfrom%2B2020-06-21%2B12-58-33.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="473" data-original-width="844" height="179" src="https://3.bp.blogspot.com/-8JXmWjGr1Ng/Xu-8AUAg_UI/AAAAAAAAC_I/WXa4-rpnd1Q8vjLPFDmei8vgOdgWLjxqACLcBGAsYHQ/s320/Screenshot%2Bfrom%2B2020-06-21%2B12-58-33.png" width="320" /></a></div></li>
</ol><h2>Related webpages</h2><ul><li> <a href="https://linuxcommando.blogspot.com/2020/06/how-to-migrate-single-site-wordpress-to.html">How to migrate single site WordPress to multisite</a></li>
<li> <a href="https://linuxcommando.blogspot.com/2013/12/install-wordpress-38-on-debian-wheezy.html">Install latest WordPress on Debian Linux</a></li>
</ul><div class="blogger-post-footer"><script type="text/javascript"><!--
google_ad_client = "pub-1640070864477148";
//728x90, created 11/11/07
google_ad_slot = "6009857287";
google_ad_width = 728;
google_ad_height = 90;
//--></script>
<script type="text/javascript"
src="http://pagead2.googlesyndication.com/pagead/show_ads.js">
</script></div>Peter Leunghttp://www.blogger.com/profile/05589860210899238688noreply@blogger.com1tag:blogger.com,1999:blog-8032022811235182759.post-78524216333392350262020-06-14T13:27:00.001-07:002020-06-23T12:25:12.319-07:00How to migrate single site WordPress to multisite<h2>The scenario</h2><div class="separator" style="clear: both; text-align: center;"><a href="https://4.bp.blogspot.com/-idT9sxMdFxU/XtkeZhLT89I/AAAAAAAAC7A/zIuLNpiSmwA8ATqDC7ofco1FPrbh3P2WACLcBGAsYHQ/s1600/wordpress-3446158_640.jpg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://4.bp.blogspot.com/-idT9sxMdFxU/XtkeZhLT89I/AAAAAAAAC7A/zIuLNpiSmwA8ATqDC7ofco1FPrbh3P2WACLcBGAsYHQ/s200/wordpress-3446158_640.jpg" width="200" height="113" data-original-width="640" data-original-height="360" /></a></div><p>I had an existing WordPress website, say <em>example1.com</em>, that was hosted on a <a href="https://www.digitalocean.com/?refcode=fda198b040e5">DigitalOcean</a> VPS running LEMP(Linux, NGINX, MySQL, PHP). The website supported HTTPS using a TLS certificate issued by <a href="https://letsencrypt.org/">Let's Encrypt</a>. </p><p>I wanted to start a new WordPress website, say <em>example2.com</em>. Barring a miracle, <em>example2.com</em> would initially have minimal traffic. </p><p>I decided that the new site would run on a virtual host on the same VPS, using WordPress's multisite feature.</p><h2>WordPress multisite</h2><p><div><a href="https://4.bp.blogspot.com/-XPumAMh56Bg/Xtkc8qGhBKI/AAAAAAAAC60/QxsgLAfiPNgabyXwTkFJHKw-MifdMl_ggCLcBGAsYHQ/s1600/wordpress-973439_640.jpg" imageanchor="1" style="clear: left; float: right; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://4.bp.blogspot.com/-XPumAMh56Bg/Xtkc8qGhBKI/AAAAAAAAC60/QxsgLAfiPNgabyXwTkFJHKw-MifdMl_ggCLcBGAsYHQ/s200/wordpress-973439_640.jpg" width="200" height="112" data-original-width="640" data-original-height="359" /></a></div>Hosting multiple websites/domains on the same VPS can be a double-edged sword. Suffice to say, the advantage is economy of scale, and the disadvantage, putting all one's eggs in one basket.</p><p>The multisite model in WordPress can be summarized as '1 instance, 1 database'. The multiple websites share the same WordPress <em>DocumentRoot</em> directory (<em>/var/www/example1</em>) and the same WordPress mySQL database.</p><p>Within the single database, site-specific information is stored in tables identified by the blog IDs. For instance, the <em>wp_posts</em> table for <em>example1.com</em> retains the same name in multisite. However, the corresponding table for <em>example2.com</em> is named <em>wp_2_posts</em> (the <em>2</em> in the name refers to the official blog ID).</p><p>Multisite introduces a new level of complexity in <a href="https://wordpress.org/support/article/multisite-network-administration/">administration</a>. Seeing multisite in action is the best way to know what you are getting into before actually migrating your production website. </p><h2>Trialing the migration</h2><div class="separator" style="clear: both; text-align: center;"><a href="https://2.bp.blogspot.com/-MsrauEuEblU/XtqIHtBuu6I/AAAAAAAAC7M/viX7vU0wnNsZJZt8Kg-fPSBzDYzEivT3QCLcBGAsYHQ/s1600/wordpress-1415619_640.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://2.bp.blogspot.com/-MsrauEuEblU/XtqIHtBuu6I/AAAAAAAAC7M/viX7vU0wnNsZJZt8Kg-fPSBzDYzEivT3QCLcBGAsYHQ/s200/wordpress-1415619_640.png" width="200" height="200" data-original-width="640" data-original-height="640" /></a></div><p>Setting up a separate VPS with the same configuration as the production system is the best option for conducting a trial migration. Notwithstanding, I opted for a poor man's platform to test the migration, my home workstation.<br />
<p>I won't be able to completely replicate at home the production environment. Most notably, no HTTPS for the home machine because there won't be TLS certificates.<br />
<p>To reflect the change from HTTPS to HTTP, I modify 2 WordPress administrative options, <em>siteurl</em> and <em>home</em>. Run the following SQL commands under mySQL:<br />
<pre><code>update wp_options set option_value='<a href="http://example1a.com">http://example1a.com</a>' where option_name = 'siteurl';
update wp_options set option_value='<a href="http://example1a.com">http://example1a.com</a>' where option_name = 'home';
</pre></code></p><p>Note that instead of reusing the same names for <em>example1.com</em> and <em>example2.com</em>, I renamed them to say <em>example1a.com</em> and <em>example2a.com</em> respectively. In addition, I configure local DNS on my home workstation to map <em>example1a.com</em> and <em>example2a.com</em> to localhost's IP address. The reason is that I can access both the production and the trial websites at the same time. Add the following lines to the <em>/etc/hosts</em> file:<br />
<pre><code>127.0.1.1 example1a.com
127.0.1.1 example2a.com
</code></pre></p><p><p>The rest of this post will detail the steps to convert WordPress from hosting a single site to hosting multiple sites.</p><h2>Configuring system</h2><div class="separator" style="clear: both; text-align: center;"><a href="https://2.bp.blogspot.com/-_E4hQKG6Dl8/Xt7h-ZfxzSI/AAAAAAAAC78/6XByVa9MVdYHjoPDy19VDAy0e6V7Ryc7wCLcBGAsYHQ/s1600/https-3344700_640.jpg" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" src="https://2.bp.blogspot.com/-_E4hQKG6Dl8/Xt7h-ZfxzSI/AAAAAAAAC78/6XByVa9MVdYHjoPDy19VDAy0e6V7Ryc7wCLcBGAsYHQ/s200/https-3344700_640.jpg" width="200" height="80" data-original-width="640" data-original-height="256" /></a></div><ol><li> Configure DNS.<br />
<p>Register <i>example2.com</i> with a domain name registry and add the appropriate DNS records. </li>
<li> Obtain Let’s Encrypt TLS certificate for second domain.<br />
<p>I assume <i>certbot</i> is already installed, <i>certbot.timer</i> enabled, and port 443 open.<br />
<p>Although it is possible to bundle multiple domains, <i>example1.com</i> and <i>example2.com</i> in a single certificate, it is recommended that you create separate certificates for unique domain names.<br />
<pre><code>$ sudo certbot certonly --webroot -n --agree-tos -m sysadmin@example2.com -w /var/www/example1.com -d example2.com
</pre></code><br />
<p><i>-m</i>: the email address for the certificate contact.<br />
<p><i>-w</i>: the <i>DocumentRoot</i> for <i>example2.com</i> which is the same as <i>example1.com</i>.<br />
<p><i>-d</i>: the domain.</li>
<li> Install certificate.<br />
<p>Link the private key and the certificate generated by Let’s Encrypt to their respective expected TLS locations.<br />
<pre><code>$ sudo ln -s /etc/letsencrypt/live/example2.com/privkey.pem /etc/ssl/private/key2.pem
$ sudo ln -s /etc/letsencrypt/live/example2.com/fullchain.pem /etc/ssl/certs/cert2.pem
</pre></code><br />
<p>Note that the names <i>key2.pem</i> and <i>cert2.pem</i> must be different from their counterparts for <i>example1.com</i>. Make a note of their names as you will need them later.</li>
<li> Configure website.<br />
<p>Create <i>/etc/nginx/sites-available/example2.com.conf</i> by copying <i>example1.com.conf</i> and making the necessary changes.<br />
<p>My skeleton <i>example2.com.conf</i> file looks like the following. The highlighted lines are relevant to the migration per se.<br />
<pre><code>server {
listen 80;
return 301 https://$host$request_uri;
}
server {
listen 443 ssl;
<font color=red>ssl_certificate /etc/ssl/certs/cert2.pem;
ssl_certificate_key /etc/ssl/private/key2.pem;
root /var/www/example1.com;
server_name example2.com *.example2.com;</font>
index index.html index.php;
location / {
try_files $uri $uri/ /index.php?q=$uri&$args;
}
location ~ \.php$ {
<font color=red>fastcgi_pass unix:/var/run/php/php7.3-fpm_example1.com.sock; </font>
}
location ~* /(?:uploads|files)/.*\.php$ {
deny all;
}
location = /robots.txt {
allow all;
log_not_found off;
access_log off;
}
add_header X-XSS-Protection "1; mode=block";
add_header X-Content-Type-Options nosniff;
add_header X-Robots-Tag none;
add_header X-Download-Options noopen;
add_header X-Permitted-Cross-Domain-Policies none;
add_header Strict-Transport-Security "max-age=15768000; includeSubDomains; preload";
add_header Referrer-Policy no-referrer;
add_header X-Frame-Options "SAMEORIGIN";
}</pre></code></li>
<p>Notes: <ul><li>The location of the certificate (<i>cert2.pem</i>) and key (<i>key2.pem</i>) need to be specified.</li>
<li>The <i>DocumentRoot</i> location is the same as <i>example1.com</i>.</li>
<li>The relevant server names are specified for this website(<i>example2.com</i>)</li>
<li>PHP handling is listening to the same socket as <i>example1.com</i> (<i>/var/run/php/php7.3-fpm_example1.com.sock</i>).</li>
</ul><li> Enable website.<br />
<pre><code>$ sudo ln -s /etc/nginx/sites-available/example2.com.conf /etc/nginx/sites-enabled/example2.com.conf</pre></code></li>
<li> Reload NGINX.<br />
<p>Test the syntax of the file changes before actually reloading the configuration files.<br />
<pre><code>$ sudo nginx -t
$ sudo systemctl reload nginx
</pre></code></li>
</ol><h2>Configuring WordPress</h2><div class="separator" style="clear: both; text-align: center;"><a href="https://4.bp.blogspot.com/-fm5qv8U1aeE/Xt7ZuTlSDRI/AAAAAAAAC7w/fq9l_0QRRWwm3UPQlX7R9XJVX7AlMlzowCLcBGAsYHQ/s1600/wordpress-3090999_640.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://4.bp.blogspot.com/-fm5qv8U1aeE/Xt7ZuTlSDRI/AAAAAAAAC7w/fq9l_0QRRWwm3UPQlX7R9XJVX7AlMlzowCLcBGAsYHQ/s200/wordpress-3090999_640.png" width="196" height="200" data-original-width="627" data-original-height="640" /></a></div><ol><li> Install <i>wp_cli</i>.<br />
<p>Although one can handcraft the necessary lines in the WordPress configuration file (<i>/var/www/example1.com/wp-config.php</i>), I’d recommend using the command-line program [wp-cli](https://wp-cli.org/). To install, run this command sequence:<br />
<pre><code>$ curl -O https://raw.githubusercontent.com/wp-cli/builds/gh-pages/phar/wp-cli.phar
$ chmod +x wp-cli.phar
$ sudo mv wp-cli.phar /usr/local/bin/wp</pre></code></li>
<li>Convert to multisite.<br />
<pre><code>$ wp core multisite-convert --subdomains --path=/var/www/example1.com
</pre></code><br />
<p>The above command can be run while the website is up because it only statically inserts the following lines into <i>wp-config.php</i>.<br />
<pre><code>define( 'WP_ALLOW_MULTISITE', true );
define( 'MULTISITE', true );
define( 'SUBDOMAIN_INSTALL', true );
$base = '/';
define( 'DOMAIN_CURRENT_SITE', 'example1.com' );
define( 'PATH_CURRENT_SITE', '/' );
define( 'SITE_ID_CURRENT_SITE', 1 );
define( 'BLOG_ID_CURRENT_SITE', 1 );
</pre></code></li>
<li> Patch ‘blocked cookie’ bug.<br />
<p>Unless the bug is patched, login to your new website (<i>example2.com</i>) is prevented. The error message from Firefox is ‘Cookies are blocked or not supported by your browser. You must enable cookies to use WordPress.’<br />
<p>To patch, edit <i>/var/www/example1.com/wp-config.php</i>, and insert the following <i>define</i> statement anywhere above the “That’s all” comment line.<br />
<pre><code>define('COOKIE_DOMAIN', false);
...
/* That's all, stop editing! Happy publishing. */
</pre></code></li>
<li> Restart the PHP-FPM and NGINX daemons.<br />
<pre><code>$ sudo systemctl restart php7.3-fpm
$ sudo systemctl restart nginx
</pre></code></li>
</ol><h2>Creating new site</h2><ol><li>Login to the original WordPress website <i>example1.com</i> using the URL(<i>https://example1.com/wp-login.php</i>).<br />
<p>The ID to use to login is the same admin ID for <i>example1.com</i>. In multisite, this admin ID is promoted to super-admin status, capable of administering all domains within the network. </li>
<div class="separator" style="clear: both; text-align: center;"><a href="https://4.bp.blogspot.com/-TZm9k-K8-mw/XuWENY5Bb8I/AAAAAAAAC8Y/NGMKcgc8rLs2U243Mi8rUE5bfcQwROXYQCLcBGAsYHQ/s1600/mysites.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://4.bp.blogspot.com/-TZm9k-K8-mw/XuWENY5Bb8I/AAAAAAAAC8Y/NGMKcgc8rLs2U243Mi8rUE5bfcQwROXYQCLcBGAsYHQ/s320/mysites.jpg" width="320" height="242" data-original-width="420" data-original-height="318" /></a></div><li> Click <i>My Sites</i> and select <i>Network Admin</i> and then <i>Sites</i>.</li>
<div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-kLUYeIG2t0Y/XuWQWk7dDUI/AAAAAAAAC8k/ujTPn55s_aoT63zhkplHhR0spendIn9ZgCLcBGAsYHQ/s1600/mysites2.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://1.bp.blogspot.com/-kLUYeIG2t0Y/XuWQWk7dDUI/AAAAAAAAC8k/ujTPn55s_aoT63zhkplHhR0spendIn9ZgCLcBGAsYHQ/s320/mysites2.jpg" width="320" height="313" data-original-width="397" data-original-height="388" /></a></div><li>Click <i>Add New</i>.</li>
<div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-txfcjs7OM7M/XuWX7IiyXbI/AAAAAAAAC8w/30TXsCV63GUtdFMvBsD02hljQPJ16vghACLcBGAsYHQ/s1600/mysites3.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://1.bp.blogspot.com/-txfcjs7OM7M/XuWX7IiyXbI/AAAAAAAAC8w/30TXsCV63GUtdFMvBsD02hljQPJ16vghACLcBGAsYHQ/s320/mysites3.jpg" width="320" height="243" data-original-width="756" data-original-height="575" /></a></div><li>Enter the required data, and click <i>Add Site</i>.</li>
<p>The <i>Site Address</i> is where one is supposed to enter the URL except it expects a subdomain, such as <i>example2</i> which it will then concatenate with the primary domain to become <i>example2.example1.com</i>, not what I really wanted as in <i>https://example2.com/</i>. So for now, I simply play along by entering <i>example2</i>, and I will change it later. If you know a better way, please let me know in a comment. <p>The <i>Admin Email</i> can be that of an existing user, say the super-admin, or a new user.
<li>Again, click <i>My Sites</i> and select <i>Network Admin</i> and then <i>Sites</i>.</li>
<div class="separator" style="clear: both; text-align: center;"><a href="https://4.bp.blogspot.com/-7JsjyK0WDYE/XuWp09i--qI/AAAAAAAAC88/NDwOFLfwlXsGAiPeEhrwA8o9V9kmE4vVwCLcBGAsYHQ/s1600/mysites4.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://4.bp.blogspot.com/-7JsjyK0WDYE/XuWp09i--qI/AAAAAAAAC88/NDwOFLfwlXsGAiPeEhrwA8o9V9kmE4vVwCLcBGAsYHQ/s320/mysites4.jpg" width="320" height="171" data-original-width="954" data-original-height="510" /></a></div><li>Hover over <i>example2.example1.com</i> and click <i>Edit</i>.</li>
<div class="separator" style="clear: both; text-align: center;"><a href="https://2.bp.blogspot.com/-7rUgEfBWw-E/XuaH5_YH4YI/AAAAAAAAC9c/TkWvIHCKCeEpAhdO1W8UjD6LrMN_43flgCLcBGAsYHQ/s1600/multi8.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://2.bp.blogspot.com/-7rUgEfBWw-E/XuaH5_YH4YI/AAAAAAAAC9c/TkWvIHCKCeEpAhdO1W8UjD6LrMN_43flgCLcBGAsYHQ/s320/multi8.jpg" width="320" height="276" data-original-width="686" data-original-height="591" /></a></div><li>Enter the correct <i>Site Address</i>, <i>https://example2.com/</i>, and click <i>Save Changes</i>.</li>
</ol><p>Now, the new website is created and ready for you to edit. Browse to <em><a href="https://example2.com/wp-login.php">https://example2.com/wp-login.php</a></em> and login as the super-admin user.</p><h2>Related webpages</h2><ul><li> <a href="https://linuxcommando.blogspot.com/2020/06/adding-google-analytics-tracking-code.html">Adding Google Analytics tracking code to a WordPress website</a></li>
<li> <a href="https://linuxcommando.blogspot.com/2013/12/install-wordpress-38-on-debian-wheezy.html">Install latest WordPress on Debian Linux</a></li>
</ul><br />
<div class="blogger-post-footer"><script type="text/javascript"><!--
google_ad_client = "pub-1640070864477148";
//728x90, created 11/11/07
google_ad_slot = "6009857287";
google_ad_width = 728;
google_ad_height = 90;
//--></script>
<script type="text/javascript"
src="http://pagead2.googlesyndication.com/pagead/show_ads.js">
</script></div>Peter Leunghttp://www.blogger.com/profile/05589860210899238688noreply@blogger.com1tag:blogger.com,1999:blog-8032022811235182759.post-14432953721687389822020-05-26T20:11:00.000-07:002020-05-26T20:11:31.846-07:00Gromit-MPX: a nifty videoconference screen annotation tool<div class="separator" style="clear: both; text-align: center;"><a href="https://2.bp.blogspot.com/-cJmwuC962G8/Xsv6yd_RQ5I/AAAAAAAAC5Q/uT5JJWTmnLYpK3naJ5KK5E_u3KrILnSYwCLcBGAsYHQ/s1600/online-learning-5163107_640.jpg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://2.bp.blogspot.com/-cJmwuC962G8/Xsv6yd_RQ5I/AAAAAAAAC5Q/uT5JJWTmnLYpK3naJ5KK5E_u3KrILnSYwCLcBGAsYHQ/s200/online-learning-5163107_640.jpg" width="200" height="133" data-original-width="640" data-original-height="427" /></a></div><p>The rise of the COVID-19 pandemic propels videoconferencing to the stratosphere of user adoption. Almost overnight, the previously unknown app <a href="https://zoom.us/">Zoom</a> became a household technology name. Technology behemoths like Google and Microsoft scrambled to beef up their own videoconferencing products to match Zoom's success. <br />
<p>Zoom allows the meeting presenter to share their desktop with other participants. Google Meet and Skype also have that screen sharing feature. What Zoom offers, as of today, but not Google Meet or Skype, is the ability to annotate the shared screen in real time.<br />
<div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-4TxUX-4HREM/Xsv0O8SWwYI/AAAAAAAAC5E/0wP6x85qKWcVKYakYZ05FQVioVy8Ud6UwCLcBGAsYHQ/s1600/board-2300038_640.jpg" imageanchor="1" style="clear: left; float: right; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://1.bp.blogspot.com/-4TxUX-4HREM/Xsv0O8SWwYI/AAAAAAAAC5E/0wP6x85qKWcVKYakYZ05FQVioVy8Ud6UwCLcBGAsYHQ/s200/board-2300038_640.jpg" width="200" height="133" data-original-width="640" data-original-height="426" /></a></div><p>Undoubtedly, Google and Microsoft will eventually incorporate screen annotation in their respective products, but for the time being, <a href="https://github.com/bk138/gromit-mpx">gromit-mpx</a> is a viable stopgap solution.<br />
<p>With gromit-mpx, presenters can annotate their desktop using free-hand drawing. It is true that Zoom as well as several third-party open-source annotation apps such as <a href="https://code.google.com/archive/p/ardesia/">ardesia</a> and <a href="http://pascal.peter.free.fr/pylote-en.html">pylote</a> give presenters more bells and whistles, for instance, to draw geometrical shapes such as solid or dashed lines and to enter text. Yet the no-frills gromit-mpx is tailor-made for videoconferencing because of its non-obtrusive, hotkey-based mode of operation.<br />
<p>In contrast to Zoom and pylote, gromit-mpx does not have a toolbar, thus saving valuable screen space. In lieu of a toolbar, gromit-mpx functionalities can be activated using hotkeys(see the table below). The inconspicuous use of hotkeys is generally less obtrusive to the presentation than the clicking of the mouse on a protruding toolbar.<br />
<table style='width: 100%;border="1";text-align: left'><thead>
<tr>
<th><span style="color: red;">Hotkey combo</span></th> <th><span style="color: red;">Corresponding action</span></th>
</tr>
</thead>
<tbody>
<tr>
<td>F9</td>
<td>Toggle drawing</td>
</tr>
<tr>
<td>Shift-F9</td>
<td>Clear screen</td>
</tr>
<tr>
<td>Ctrl-F9</td>
<td>Toggle visibility</td>
</tr>
<tr>
<td>Alt-F9</td>
<td>Quit app</td>
</tr>
<tr>
<td>Click</td>
<td>Draw with red pen (default)</td>
</tr>
<tr>
<td>Shift-Click</td>
<td>Blue pen </td>
</tr>
<tr>
<td>Ctrl-Click</td>
<td>Yellow pen</td>
</tr>
<tr>
<td>Wheel-button click</td>
<td>Green pen with arrow</td>
</tr>
<tr>
<td>Right click</td>
<td>Eraser</td>
</tr>
</tbody>
</table></p><h2>Installation</h2><p>To install gromit-mpx on Debian or Ubuntu, enter:<br />
<p><code><br />
# apt install gromit-mpx<br />
</code></p><h2>Conclusion</h2><p>If a videoconference presenter has the most basic requirement for an annotation tool, for instance, to draw meeting participants' attention to an area of the screen, gromit-mpx fits the bill well. Its handy hotkeys make annotation more seamless and speedy than the clunky toolbar used by more feature-complete apps, even Zoom. </p><div class="blogger-post-footer"><script type="text/javascript"><!--
google_ad_client = "pub-1640070864477148";
//728x90, created 11/11/07
google_ad_slot = "6009857287";
google_ad_width = 728;
google_ad_height = 90;
//--></script>
<script type="text/javascript"
src="http://pagead2.googlesyndication.com/pagead/show_ads.js">
</script></div>Peter Leunghttp://www.blogger.com/profile/05589860210899238688noreply@blogger.com0tag:blogger.com,1999:blog-8032022811235182759.post-59737858711486767152020-05-14T13:07:00.001-07:002020-05-14T13:07:05.396-07:00Joplin vs Orgzly as note-taking to-do apps<div class="separator" style="clear: both; text-align: center;"><a href="https://3.bp.blogspot.com/-OCAvAGUK-TA/WwnP2vLj_mI/AAAAAAAACk0/F9SO4QNM2xM6eQCTt6fgjx8RC_f7IYuEwCLcBGAs/s1600/writing-3383287_1280.jpg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="780" data-original-width="1280" height="195" src="https://3.bp.blogspot.com/-OCAvAGUK-TA/WwnP2vLj_mI/AAAAAAAACk0/F9SO4QNM2xM6eQCTt6fgjx8RC_f7IYuEwCLcBGAs/s320/writing-3383287_1280.jpg" width="320" /></a></div><p>This post evaluates 2 note-taking, to-do list managers: <a href="https://joplinapp.org/">Joplin</a> and <a href="https://orgmode.org/#docs">org-mode</a>/<a href="http://www.orgzly.com/help">Orgzly</a>. Both are free, open-source, cross-platform software.</p><p>As an avid emacs user, I have always used <a href="https://orgmode.org/#docs">org-mode</a> on my Linux desktop to take notes and compile my to-do lists. So much so, I held out as long as possible before I switched to another tool that could actually run on Android. Org-mode as an emacs tool did not support Android at the time. Painful as it was, I replaced org-mode with the cross-platform tool Joplin. </p><p><a href="https://joplinapp.org/">Joplin</a> served me very well on both Linux and Android platforms…until I came across an Android app named Orgzly. Org-mode and Orgzly share the same plain text file format, and according to <a href="http://www.orgzly.com/help#org-mode">Orgzly documentation</a>, 'files generated by Orgzly might differ in the amount of white space … Any other difference would be considered a serious bug.' File compatibility means that you can edit your tasks and notes using org-mode on Linux and Orgzly on Android.</p><p>One key difference between org-mode and Orgzly is how you edit the shared underlying files. Using org-mode, you edit the text files directly inside emacs the text editor. In contrast, you use Orgzly's GUI for editing.</p><p>Below, I compare Joplin and org-mode/Orgzly. <br />
<h2>Portability</h2><div class="separator" style="clear: both; text-align: center;"><a href="https://4.bp.blogspot.com/-TILj994nRCs/XryDYVqaYJI/AAAAAAAAC3U/kkzJ4h0uKp4HOunImnodvsElVFlsPG5NwCLcBGAsYHQ/s1600/tablet-313002_640.jpg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://4.bp.blogspot.com/-TILj994nRCs/XryDYVqaYJI/AAAAAAAAC3U/kkzJ4h0uKp4HOunImnodvsElVFlsPG5NwCLcBGAsYHQ/s200/tablet-313002_640.jpg" width="200" height="143" data-original-width="640" data-original-height="459" /></a></div>You can run Joplin in 3 ways: <br />
<ol><li>as a desktop app on Linux, Windows or macOS,<br />
<li>as a mobile app on Android or iOS, and<br />
<li>as a terminal program on Linux, FreeBSD, macOS or Windows.<br />
</ol>I used both its Linux desktop version as well as the Android mobile version, and had no problem vouching for Joplin.</p><p>Orgzly runs on Android only (no iOS version yet). For non-mobile platforms, you will need to run org-mode within the emacs editor. Although it has been done before, converting to emacs just to use org-mode may be an overkill for most people.</p><p>On portability, Joplin has a definite advantage over org-mode/Orgzly.
<h2>Installability</h2></p><div class="separator" style="clear: both; text-align: center;"><a href="https://4.bp.blogspot.com/-U1PbFHc06O0/Xrsuz3-vSeI/AAAAAAAAC28/oV9raiHNIpQeVIYTlsESO5kDVSd81WHggCLcBGAsYHQ/s1600/progress-29044_640.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://4.bp.blogspot.com/-U1PbFHc06O0/Xrsuz3-vSeI/AAAAAAAAC28/oV9raiHNIpQeVIYTlsESO5kDVSd81WHggCLcBGAsYHQ/s200/progress-29044_640.png" width="200" height="100" data-original-width="640" data-original-height="320" /></a></div><p>Mobile versions of Joplin can be installed via the respective Google and Apple app stores. Installing it on a desktop (Linux, Windows, macOS) is just as convenient. Joplin is available to download from the standard repository of many Linux distributions including Debian. You can also download the AppImage version on <a href="https://joplinapp.org/">Joplin's website</a>.</p><p>Orgzly can be installed on Android using either Google Play or F-Droid, the repository for free and open-source Android apps. Org-mode, the desktop counterpart, is now part of emacs: installing emacs automatically installs org-mode. </p><p>It is a tie.
<h2>Data import/export</h2><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-PZ-Ox6ay2Bw/Xr2kt4h5hQI/AAAAAAAAC4A/rRdgfz1iHBcxwLc9LDIgnhmbe9qSDybogCLcBGAsYHQ/s1600/arrow2-2085195_640.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://1.bp.blogspot.com/-PZ-Ox6ay2Bw/Xr2kt4h5hQI/AAAAAAAAC4A/rRdgfz1iHBcxwLc9LDIgnhmbe9qSDybogCLcBGAsYHQ/s200/arrow2-2085195_640.png" width="200" height="133" data-original-width="640" data-original-height="426" /></a></div>Unless you are using a note-taking to-do app for the first time, you will want to easily import any data from your existing app into the new app. Conversely, to prevent vendor lock-in to any 1 app, you want to be able to export data in a format that other apps can easily import. One such format is ENEX, the file format for Evernote, the app with arguably the largest installed user base.</p><p>Joplin can import ENEX files, but cannot export to the same. In recompense, you can import and export data in Markdown, PDF and JSON formats.</p><p>Orgzly currently only supports the import of org-mode files, and does not support any import or export of third-party file formats. </p><p>The clear winner is Joplin.
<h2>User-friendliness</h2><div class="separator" style="clear: both; text-align: center;"><a href="https://4.bp.blogspot.com/-WNNilM9Q4b0/XrsuEyfPy2I/AAAAAAAAC20/U1-II-vjKZMT2ZkUv6m3zKgNVwpw3A0iwCLcBGAsYHQ/s1600/experience-3239623_640.jpg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://4.bp.blogspot.com/-WNNilM9Q4b0/XrsuEyfPy2I/AAAAAAAAC20/U1-II-vjKZMT2ZkUv6m3zKgNVwpw3A0iwCLcBGAsYHQ/s200/experience-3239623_640.jpg" width="200" height="130" data-original-width="640" data-original-height="417" /></a></div>Both Joplin and Orgzly are minimalistic (even spartan in the case of Orgzly) but highly functional in their user interface design. </p><p>To their credit, both offer dark mode, i.e., the ability to set the background to dark. </p><p>Joplin has a slight edge over Orgzly in aesthetics.</p><p>Winner: Joplin.</p><h2>Cloud storage</h2><div class="separator" style="clear: both; text-align: center;"><a href="https://4.bp.blogspot.com/-QS5_7fLq2s8/Xrsaa2QNSrI/AAAAAAAAC2c/K6nATxxHRW4ufEW5WbyPMvf8gPBrcB8ZwCLcBGAsYHQ/s1600/cloud-4273197_640.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://4.bp.blogspot.com/-QS5_7fLq2s8/Xrsaa2QNSrI/AAAAAAAAC2c/K6nATxxHRW4ufEW5WbyPMvf8gPBrcB8ZwCLcBGAsYHQ/s320/cloud-4273197_640.png" width="241" height="320" data-original-width="481" data-original-height="640" /></a></div><p>Both Joplin and Orgzly support data storage on popular cloud platforms. Cloud storage enables you to access your tasks and notes from multiple devices and platforms.</p><p>Some cloud platforms provide custom API that client apps such as Joplin and Orgzly can use to make connections. There is a cost to using API: the client app needs to write custom code for each cloud service.</p><p>The advent of the WebDAV protocol has created a level playing field for apps that need to exchange data over the Internet. Client apps only need to write the WebDAV interface once to support access to all WebDAV-compliant cloud services.</p><p>Joplin supports Dropbox and OneDrive via native API, and Nextcloud via the WebDAV protocol. Note that with OneDrive, upload files are restricted to a maximum size of 4 MB, which is relatively low if you are attaching large multimedia files.</p><p>Orgzly supports Dropbox through native API and Nextcloud and other cloud services through WebDAV.</p><p>A tie.
<h2>Encryption</h2><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-1b6_bgMlsWY/Xry5x1N20qI/AAAAAAAAC3g/3a_SBeUxYKs66zFCSS2BhRALwc-dtXWRACLcBGAsYHQ/s1600/encrypted-156514_640.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://1.bp.blogspot.com/-1b6_bgMlsWY/Xry5x1N20qI/AAAAAAAAC3g/3a_SBeUxYKs66zFCSS2BhRALwc-dtXWRACLcBGAsYHQ/s200/encrypted-156514_640.png" width="183" height="200" data-original-width="585" data-original-height="640" /></a></div>As discussed above, both apps can store data in the cloud. Data is regularly transmitted in the cloud to keep client apps synchronized.</p><p>Joplin data transmission is <a href="https://joplinapp.org/e2ee/">encrypted</a> end-to-end; Orgzly, unencrypted.</p><p>With Joplin, your privacy is protected: not even Joplin developers or your cloud host such as Dropbox can access your encrypted data. </p><p>This may well be the killer feature that swings the pendulum all the way in favor of Joplin for most people.</p><h2>Project viability</h2><div class="separator" style="clear: both; text-align: center;"><a href="https://3.bp.blogspot.com/-p2JGbKApsrY/Xr2j5eKbeQI/AAAAAAAAC30/SP8PyghiTIkIcTv1ZfwvEisHo3G7br-VwCLcBGAsYHQ/s1600/alien2-1292972_640.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://3.bp.blogspot.com/-p2JGbKApsrY/Xr2j5eKbeQI/AAAAAAAAC30/SP8PyghiTIkIcTv1ZfwvEisHo3G7br-VwCLcBGAsYHQ/s200/alien2-1292972_640.png" width="139" height="200" data-original-width="445" data-original-height="640" /></a></div><p>The 2 projects are very similar on this point – both stable and actively maintained by a team of one major developer. So, don't expect major new features every few months. </p><p>For mature applications like note-taking to-do list managers, that may actually be a benefit.</p><p>According to Google Play, both apps are downloaded roughly the same number of times(50,000+). Their numbers pale in comparison to the behemoth Evernote (100,000,000+). However, both projects have reached a critical mass in their respective user communities.</p><p>A tie.
<h2>Conclusion</h2>Both Joplin and org-mode/Orgzly are more than capable to do the basic job. But Joplin is the eminently obvious overall winner … unless you are a hardcore emacs fan. </p><br />
<div class="blogger-post-footer"><script type="text/javascript"><!--
google_ad_client = "pub-1640070864477148";
//728x90, created 11/11/07
google_ad_slot = "6009857287";
google_ad_width = 728;
google_ad_height = 90;
//--></script>
<script type="text/javascript"
src="http://pagead2.googlesyndication.com/pagead/show_ads.js">
</script></div>Peter Leunghttp://www.blogger.com/profile/05589860210899238688noreply@blogger.com0tag:blogger.com,1999:blog-8032022811235182759.post-22323008271765387112020-04-25T16:45:00.000-07:002020-04-25T16:45:23.555-07:00How to remove passwords from PDF files<div class="separator" style="clear: both; text-align: center;"><a href="https://3.bp.blogspot.com/-ceykUcVxWW8/XpnjemF9DSI/AAAAAAAAC1g/H_7AuB8A1DkIXu_ZT90c0-sAgEoNV-abwCLcBGAsYHQ/s1600/pdf-155498_640.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://3.bp.blogspot.com/-ceykUcVxWW8/XpnjemF9DSI/AAAAAAAAC1g/H_7AuB8A1DkIXu_ZT90c0-sAgEoNV-abwCLcBGAsYHQ/s200/pdf-155498_640.png" width="200" height="200" data-original-width="640" data-original-height="640" /></a></div><p>Recently, a financial institution emailed me a password-protected PDF file. Handling such PDFs was a nuisance. First, I had to call them to obtain the password. Second, because that was a file I'd like to access in the future, I needed to record the password, unless … I could somehow remove the password from the PDF file.</p><p>This post outlines several ways to remove a password from a PDF. </p><h2>pdftk</h2><p><b>pdftk</b> is my go-to tool for manipulating PDFs. To save a password-protected PDF into a new file, without the password, simply execute a command like this:</p><pre>$ pdftk <i>MyInput.pdf</i> input_pw <i>PASSWORD</i> output <i>MyOutput.pdf</i>
WARNING: The creator of the input PDF:
MyInput.pdf
has set an owner password (which is not required to handle this PDF).
You did not supply this password. Please respect any copyright.
</pre><p>You can safely ignore the warning message. </p><p>An encrypted PDF file can have up to 2 passwords, the user password and the owner password, with the latter having more privileges. Either password will let you perform the operation, although <strong>pdftk</strong> prefers the owner password if the PDF has one. If you did not create the original PDF, most likely, the password given to you was the user password. Hence the warning.</p><p>Security conscious users would balk at specifying the plain-text password on the command line. Specifying the <strong>do_ask</strong> parameter allows you to enter the password via standard input.<br />
<pre>$ pdftk <i>MyInput.pdf</i> output <i>MyOutput.pdf</i> do_ask
</pre></p><h2>qpdf</h2><p>An alternative solution is to use <strong>qpdf</strong>. For instance,<br />
<pre><code>
$ qpdf --decrypt --password=<i>PASSWORD</i> -- <i>MyInput.pdf</i> <i>MyOutput.pdf</i>
</code>
</pre></p><p>Note that the marker <strong>--</strong> is used to separate the options from the input and output filenames.</p><p>To hide the password from the command line, specify the <strong>@-</strong> argument, which enables you to enter arguments via standard input. When prompted, enter <strong>--password=<i>PASSWORD</i></strong>.</p><pre><code>
$ qpdf --decrypt @- -- <i>MyInput.pdf</i> <i>MyOutput.pdf</i>
</code>
</pre><p>Alternatively, you can specify the password inside a file, for instance, <em>@/home/peter/arguments.txt</em>. Note that the filepath is appended to the single character <strong>@</strong>. The file contains the line <strong>--password=<i>PASSWORD</i></strong>.</p><pre><code>
$ qpdf --decrypt <i>@/home/peter/arguments.txt</i> -- <i>MyInput.pdf</i> <i>MyOutput.pdf</i>
</code>
</pre><h2>pdftops/ps2pdf</h2><p>This solution is more involved than the first 2: first convert the PDF to Postscript, and then back to PDF. I include it here to show an alternative approach, and it is probably not something you will actually do.</p><ol><li>To convert it to Postscript:<br />
<pre><code>
$ pdftops -upw <i>PASSWORD</i> <i>MyInput.pdf</i> <i>MyInput.ps</i>
</code>
</pre>Note that -<b>upw</b> refers to the user password. If you have the owner password instead, replace -<b>upw</b> with -<b>opw</b>.<br />
</li>
<li>To save it back to PDF:<br />
<pre><code>
$ ps2pdf <i>MyInput.ps</i> <i>MyOutput.pdf</i>
</code>
</pre></li>
</ol><div class="blogger-post-footer"><script type="text/javascript"><!--
google_ad_client = "pub-1640070864477148";
//728x90, created 11/11/07
google_ad_slot = "6009857287";
google_ad_width = 728;
google_ad_height = 90;
//--></script>
<script type="text/javascript"
src="http://pagead2.googlesyndication.com/pagead/show_ads.js">
</script></div>Peter Leunghttp://www.blogger.com/profile/05589860210899238688noreply@blogger.com0tag:blogger.com,1999:blog-8032022811235182759.post-54228799823276922552020-04-09T11:47:00.000-07:002020-04-09T12:15:46.898-07:00inxi: the Swiss Army knife for displaying Linux sysinfo<html><head><title></title></head><body><blockquote>Do one thing and do it well - <a href="https://en.wikipedia.org/wiki/Unix_philosophy">the Unix philosophy</a></blockquote><br />
<strong>inxi</strong> is the antithesis of the above venerable Unix philosophy. Many excellent tools exist for providing aspects of system and hardware information —<strong>lsb_release</strong>, <strong>uname</strong>, <strong>lshw</strong>, <strong>lscpu</strong>, <strong>lspci</strong>, <strong>lsusb</strong>, <strong>dmidecode</strong>, <strong>uptime</strong>, <strong>free</strong>, <strong>ip</strong>, <strong>parted</strong>, <strong>acpi</strong>, etc. Some of those tools may even report more details than <strong>inxi</strong>, but there is a definite advantage for using <strong>inxi</strong>—with just 1 command, you can see at a glance a machine's overall hardware and system configuration and real-time status. <br />
<br />
System administrators and technical support engineers work with many machines, often at the same time. <strong>inxi</strong> enables them to quickly get a broad system configuration overview and assess the current machine status before doing maintenance or troubleshooting. <br />
<br />
The tool organizes the machine data into the following categories:<br />
<ul><li>System (hostname, kernel, 64-/32-bit, desktop…)</li>
<li>Machine (model, serial #, BIOS…)</li>
<li>CPU (model, speed…)</li>
<li>Graphics</li>
<li>Audio devices</li>
<li>Network devices</li>
<li>Drives</li>
<li>Partitions</li>
<li>USB devices</li>
<li>Sensors (temperatures, fan speed…)</li>
<li>Repositories</li>
<li>Real-time status (# processes, uptime…)</li>
</ul><h2>Installation</h2><br />
To install <strong>inxi</strong> on Debian buster, <br />
<code><br />
# apt update && apt install inxi<br />
</code><br />
<br />
<h2>Dependency checking</h2><br />
<strong>inxi</strong> calls numerous helper programs to do the actual work, not all of them may be pre-installed. Run the following <strong>inxi</strong> command (as non-root user) to test what is potentially missing on your system:<br />
<br />
<code><br />
# inxi --recommends<br />
-------------------------------------------------------<br />
Test: recommended system programs:<br />
...<br />
ipmitool: -s IPMI sensors (servers)........... Missing<br />
ipmi-sensors: -s IPMI sensors (servers)....... Missing<br />
...<br />
The following recommended system programs are missing:<br />
Program: ipmitool ~ Install package: ipmitool<br />
Program: ipmi-sensors ~ Install package: freeipmi-tools<br />
...<br />
-------------------------------------------------------<br />
</code><br />
Note that the checking does not take into consideration whether your system actually supports the use of the helper programs. For instance, the 2 missing programs above (<strong>ipmitool</strong> and <strong>ipmi-sensors</strong>) only apply to servers. In this example, the target machine is not a server and does not support IPMI, so I did not install the recommended programs.<br />
<br />
You must judge the merits of installing each helper program reported missing. <br />
<br />
<h2>Usage</h2><br />
<h3>Root or non-root</h3><br />
You can run <strong>inxi</strong> as either <strong>root</strong> or a regular user. Certain output is restricted to <strong>root</strong> only, e.g., the motherboard serial number and detailed RAM data.<br />
<br />
<h3>Basics</h3><br />
Although you can run <strong>inxi</strong> without any argument to get basic CPU and memory information, I'd recommend running it with <strong>-F</strong>.<br />
<br />
<a href="https://4.bp.blogspot.com/-7OocXlErWDc/Xo9URjk1GVI/AAAAAAAAC0E/PKNneOiTjvcDzlk7vM-BvKc6OzrGS3SUgCLcBGAsYHQ/s1600/Screenshot%2Bfrom%2B2020-04-09%2B09-50-00.png" imageanchor="1"><img border="0" data-original-height="877" data-original-width="890" height="631" src="https://4.bp.blogspot.com/-7OocXlErWDc/Xo9URjk1GVI/AAAAAAAAC0E/PKNneOiTjvcDzlk7vM-BvKc6OzrGS3SUgCLcBGAsYHQ/s640/Screenshot%2Bfrom%2B2020-04-09%2B09-50-00.png" width="640" /></a><br />
<br />
<strong>-F</strong> is for <strong>F</strong>ull, and is a shorthand for specifying all uppercase letter arguments (with some exceptions). <br />
<br />
For instance, specifying <b>-F</b> automatically includes <b>-P</b>, but not <b>-p</b>. The uppercase argument <b>-P</b> shows partition information for the basic partitions: <strong>/</strong>, <strong>/boot</strong>, <strong>/home</strong>, etc. The lowercase letter argument <strong>-p</strong> includes <strong>snap</strong> partitions created when installing software using <strong>snap</strong>.<br />
<br />
<code><br />
# inxi -Fxxxz<br />
</code><br />
<br />
<strong>inxi</strong> output may contain IP addresses, MAC addresses, serial numbers—data that can uniquely identify the target system. If privacy is an issue, specify the <strong>-z</strong> flag to filter out private data from the report. Note that the default is to display the aforementioned data. <br />
<br />
You can dial up the level of details in <strong>inxi</strong> output using the <strong>-x</strong> flag. Optionally specify up to 3 increasing levels of details: <strong>-x</strong>, <strong>-xx</strong>, and <strong>-xxx</strong>.<br />
<br />
<h3>Advanced</h3><br />
If you want more details than what the <strong>-F</strong> option gives you, you can specify additional arguments to focus on specific aspects of your system. <br />
<code><br />
# inxi -Fxmip -t --usb<br />
</code><br />
<br />
The following is my favourite subset of the available arguments.<br />
<br />
<h4><strong>-d</strong></h4><br />
<strong>inxi -F</strong> only displays data about hard disk drives. To include optical/DVD drives, specify <strong>-d</strong>.<br />
<br />
<h4><strong>-i</strong></h4><br />
Default <strong>-F</strong> output hides the IP addresses for your network interfaces. To display IP, add <strong>-i</strong>. <br />
<br />
<h4><strong>-m</strong></h4><br />
The <strong>-m</strong> argument reports data about individual memory slots.<br />
<br />
<h4><strong>-p</strong></h4><br />
<strong>-F</strong> only reports standard partitions (<strong>/etc</strong>, <strong>/home</strong>, <strong>/opt</strong>…) and swap partitions. <strong>-p</strong> displays all mounted partitions, including partitions created by <strong>snap</strong>.<br />
<br />
<h4><strong>-r</strong></h4><br />
This argument reports software package repository information.<br />
<br />
<h4><strong>-t</strong></h4><br />
By default, <strong>-t</strong> displays the top 5 memory- and CPU-using processes. You can restrict to CPU or memory only, and adjust the number of processes reported. For instance, to display the top 10 memory-using processes, specify <strong>-tm10</strong>; top 10 CPU-using processes, <strong>-tc10</strong>.<br />
<br />
Separate <strong>-t</strong> from other arguments(or add it to the end of an argument chain); otherwise <strong>inxi</strong> may return a syntax error.<br />
<code><br />
# inxi -Fxmip -tc10 <br />
</code><br />
<br />
<h4><strong>--usb</strong></h4><br />
<strong>--usb</strong> displays USB device information.<br />
<br />
<h3>Make it pretty</h3><br />
You can choose a color theme for <strong>inxi</strong> output. The argument is <strong>-c</strong> followed by a value between 0 and 42 inclusive, corresponding to the color theme.<br />
<br />
<a href="https://2.bp.blogspot.com/-0ufPLDpgsIE/Xo9Z2TQkxoI/AAAAAAAAC0Q/64Ier3leH_cbyEH2he23CmP_DR1kcYmDACLcBGAsYHQ/s1600/Screenshot%2Bfrom%2B2020-04-09%2B10-21-19.png" imageanchor="1"><img border="0" data-original-height="658" data-original-width="982" height="429" src="https://2.bp.blogspot.com/-0ufPLDpgsIE/Xo9Z2TQkxoI/AAAAAAAAC0Q/64Ier3leH_cbyEH2he23CmP_DR1kcYmDACLcBGAsYHQ/s640/Screenshot%2Bfrom%2B2020-04-09%2B10-21-19.png" width="640" /></a><br />
<br />
Using the <strong>-c95</strong> argument, you can preview the list of available color themes and then select one for the current <strong>inxi</strong> command.<br />
<code><br />
# inxi -Fx -c95<br />
</code><br />
</body></html><div class="blogger-post-footer"><script type="text/javascript"><!--
google_ad_client = "pub-1640070864477148";
//728x90, created 11/11/07
google_ad_slot = "6009857287";
google_ad_width = 728;
google_ad_height = 90;
//--></script>
<script type="text/javascript"
src="http://pagead2.googlesyndication.com/pagead/show_ads.js">
</script></div>Peter Leunghttp://www.blogger.com/profile/05589860210899238688noreply@blogger.com0tag:blogger.com,1999:blog-8032022811235182759.post-69537787300858323972019-05-29T16:32:00.000-07:002019-05-30T22:17:13.164-07:00Beware of this find command gotcha <div class="separator" style="clear: both; text-align: center;">
<a href="https://3.bp.blogspot.com/-22pNonypZhg/XO8WYJmj4vI/AAAAAAAACsk/jhS-uSRfjMI0BSuXjXkMN1pVgvempl_NgCLcBGAs/s1600/database-search-2797375_640.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="640" data-original-width="640" height="200" src="https://3.bp.blogspot.com/-22pNonypZhg/XO8WYJmj4vI/AAAAAAAACsk/jhS-uSRfjMI0BSuXjXkMN1pVgvempl_NgCLcBGAs/s200/database-search-2797375_640.png" width="200" /></a></div>
<br />
<br />
<i>find</i> is a basic useful command that Linux users run all the time. The command searches a file system from a given starting location, and returns all matches based on input filters that you provide as arguments.<br />
<br />
<h2>
The Gotcha</h2>
<br />
The gotcha is when you try to narrow the search by pruning a sub-directory from the search (including the directory itself and everything under it). For instance, suppose you want to find all files under the directory <i>/data</i> that are owned by <i>root</i>, excluding the sub-directory <i>/data/keepit</i> and all files underneath.<br />
<br />
<a href="https://1.bp.blogspot.com/-aoWF_izVdBU/XO8T2USkbhI/AAAAAAAACsI/ivUsLX1szRYJih0Gj0GDjfOnpEq9RW0yQCLcBGAs/s1600/scissors-311500_640.png" imageanchor="1" style="clear: right; display: inline; float: right; margin-bottom: 1em; margin-left: 1em; text-align: center;"><img border="0" data-original-height="354" data-original-width="640" height="111" src="https://1.bp.blogspot.com/-aoWF_izVdBU/XO8T2USkbhI/AAAAAAAACsI/ivUsLX1szRYJih0Gj0GDjfOnpEq9RW0yQCLcBGAs/s200/scissors-311500_640.png" width="200" /></a><br />
<br />
My first attempt at the solution results in the following <i>find</i> command.<br />
<code> <br />
find /data -path /data/keepit -prune -o -user 0 <br />
</code> <br />
<br />
The <i>-o</i> argument specifies the logical 'or' operator. The expression on the left, '<i>-path /data/keepit -prune'</i> indicates where to prune the search. The idea is that when the search reaches <i>/data/keepit</i>, the <i>-prune </i>argument causes the search to not descend further into the sub-directory. Furthermore, <i>-prune</i> always returns true. Hence, the whole expression returns 'true', without having to evaluate the expression on the right of <i>-o</i>.<br />
<br />
The expression right of <i>-o</i> tests for <i>root</i> ownership (<i>root</i> is user 0).<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://3.bp.blogspot.com/-sSh4cnB_Pbc/XPAg_QKA7II/AAAAAAAACsw/MD50TSqEEJETfGVLKBAyHKntKIkS9mKawCLcBGAs/s1600/oops-1432954_640.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="640" data-original-width="640" height="200" src="https://3.bp.blogspot.com/-sSh4cnB_Pbc/XPAg_QKA7II/AAAAAAAACsw/MD50TSqEEJETfGVLKBAyHKntKIkS9mKawCLcBGAs/s200/oops-1432954_640.png" width="200" /></a></div>
<br />
I was befuddled to learn that running the above command returns <i>/data/keepit </i>(but not its descendants). If the search is snipped at <i>/data/keepit</i>, why is the sub-directory itself included in the output? Besides, <i>/data/keepit</i> is not owned by <i>root</i>.<br />
<br />
Being unaware of this behavior could lead to some unintended and very bad consequences as files named in the <i>find</i> output are often piped to the <i>xargs</i> command for further processing.<br />
<br />
<h2>
The Explanation</h2>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-vAW9fjPyi4A/XPBRwCQ3rMI/AAAAAAAACtE/Z_hufAo0YIsQ836Z-r8I2zSiXiJxGk--wCLcBGAs/s1600/question-2736480_640.jpg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="374" data-original-width="640" height="185" src="https://1.bp.blogspot.com/-vAW9fjPyi4A/XPBRwCQ3rMI/AAAAAAAACtE/Z_hufAo0YIsQ836Z-r8I2zSiXiJxGk--wCLcBGAs/s320/question-2736480_640.jpg" width="320" /></a></div>
<br />
Before I present my solution, let's discuss why the point of pruning, i.e., the sub-directory named in <i>-path,</i> is actually included in the output.<br />
<br />
The primary purpose of <i>find</i> is to search for file matches. Yet, it can have side effects through actions you specify on the command line. In addition to <i>-print</i>/<i>-print0</i>, there is also the <i>-exec</i> action. Unless you explicitly specify an action, the <i>find</i> command assumes the default action is <i>-print</i>.<br />
<br />
The above example has no explicit <i>-print</i> or <i>-exec</i> action, therefore, the action defaults to print all file matches. This explains why <i>/data/keepit</i>, a match for <i>-path</i>, is in the output. Its descendants, on the other hand, were excluded because of pruning.<br />
<br />
<h2>
The Solution</h2>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-zOkXRWollFE/XPBRBo1tB6I/AAAAAAAACs8/WWWXGbVFU6Ij6fY-egcJ4qTsU5xugBSGACLcBGAs/s1600/thought-2123970_640.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="444" data-original-width="640" height="222" src="https://1.bp.blogspot.com/-zOkXRWollFE/XPBRBo1tB6I/AAAAAAAACs8/WWWXGbVFU6Ij6fY-egcJ4qTsU5xugBSGACLcBGAs/s320/thought-2123970_640.jpg" width="320" /></a></div>
<br />
My solution is to specify <i>-print </i>explicitly on the command line.<br />
<br />
<code>find /data -path /data/keepit -prune -o -user root <b>-print</b><br />
</code><br />
<br />
Lo and behold. When you run the above command, <i>/data/keepit</i> is no longer part of the output.<br />
<br />
By specifying the <i>-print</i> action explicitly, the <i>find</i> command no longer defaults to printing out each file match. Instead, it will only print a file match if it is explicitly requested.<br />
<br />
<h2>
Summary & Conclusion</h2>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-uoTo1REspbg/XPCDRy7VZUI/AAAAAAAACtQ/LAg9HIX04f8-K69D2tfUqEOGdMFLGBgggCLcBGAs/s1600/check-145512_640.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="640" data-original-width="640" height="200" src="https://1.bp.blogspot.com/-uoTo1REspbg/XPCDRy7VZUI/AAAAAAAACtQ/LAg9HIX04f8-K69D2tfUqEOGdMFLGBgggCLcBGAs/s200/check-145512_640.png" width="200" /></a></div>
The pruning logic of the <i>find</i> command is quite confusing. Reading its <i>man</i> page offers some help, but may generate more questions than answers. I hope that this article is of help. But, I recommend that before you use the <i>-prune</i> feature on your production data, test it on some dummy data first.<br />
<br />
You have been forewarned.<div class="blogger-post-footer"><script type="text/javascript"><!--
google_ad_client = "pub-1640070864477148";
//728x90, created 11/11/07
google_ad_slot = "6009857287";
google_ad_width = 728;
google_ad_height = 90;
//--></script>
<script type="text/javascript"
src="http://pagead2.googlesyndication.com/pagead/show_ads.js">
</script></div>Peter Leunghttp://www.blogger.com/profile/05589860210899238688noreply@blogger.com1tag:blogger.com,1999:blog-8032022811235182759.post-33280321280194074962019-03-11T13:37:00.004-07:002019-03-11T13:37:54.240-07:00ts: epitome of the Unix philosophy<blockquote>Do one thing and do it well - <a href="https://en.wikipedia.org/wiki/Unix_philosophy">the Unix philosophy</a></blockquote><br />
In this new age of Linux bloatware (hello, <i>systemd</i>), it is exhilarating to discover small gems like <i>ts</i>, a command-line tool that prepends a timestamp to each output line.<br />
<br />
<h2>How is this useful?</h2><br />
I run scripts all the time—bash scripts, Ansible playbooks, etc–to automate system administration tasks. Many longer-running scripts that I run output statements in real-time to report what they are doing. For example, running an Ansible playbook will automatically output the name of the individual task as it is being executed. By default, however, no timestamps are displayed for the tasks. <br />
<br />
<code><br />
$ ansible-playbook -b -i hostsfile myPlaybook.yml<br />
PLAY [localhost] ****************************************************************<br />
<br />
TASK [Gathering Facts] **********************************************************<br />
ok: [localhost]<br />
<br />
TASK [Disabe Caps Lock] *********************************************************<br />
ok: [localhost]<br />
<br />
TASK [Install X apps] ***********************************************************<br />
ok: [localhost] => (item=autokey-gtk)<br />
ok: [localhost] => (item=gnucash)<br />
<br />
TASK [Install 64-bit texamker - Debian] *****************************************<br />
changed: [localhost]<br />
...snipped...<br />
PLAY RECAP **********************************************************************<br />
localhost : ok=8 changed=2 unreachable=0 failed=0 <br />
</code><br />
<br />
Often, I do want to display the timestamps for logging or troubleshooting purposes. An unusually short (or long) execution may signal something is amiss. <br />
<br />
Granted, you can use the Ansible-specific <i>profile_tasks</i> plugin to <a href="https://sketchingdev.co.uk/blog/profiling-ansible-playbooks-to-csv.html">profile your tasks</a>. But, I propose <i>ts</i> as a quick-and-dirty solution: just pipe Ansible output to <i>ts</i> like the following.<br />
<br />
<code><br />
$ ansible-playbook -b -i hostsfile myPlaybook.yml |ts<br />
Mar 11 11:44:05 PLAY [localhost] ****************************************************************<br />
Mar 11 11:44:05 <br />
Mar 11 11:44:05 TASK [Gathering Facts] **********************************************************<br />
Mar 11 11:44:05 ok: [localhost]<br />
Mar 11 11:44:06 <br />
Mar 11 11:44:06 TASK [Disable Caps Lock] *********************************************************<br />
Mar 11 11:44:06 ok: [localhost]<br />
Mar 11 11:44:06 <br />
Mar 11 11:44:06 TASK [Install X apps] ***********************************************************<br />
Mar 11 11:44:10 changed: [localhost] => (item=autokey-gtk)<br />
Mar 11 11:44:14 changed: [localhost] => (item=gnucash)<br />
Mar 11 11:44:14<br />
Mar 11 11:44:14 TASK [Install 64-bit texamker - Debian] *****************************************<br />
Mar 11 11:44:19 changed: [localhost]<br />
...snipped...<br />
Mar 11 11:44:30 PLAY RECAP **********************************************************************<br />
Mar 11 11:44:30 localhost : ok=8 changed=3 unreachable=0 failed=0 <br />
</code><br />
<br />
<h2>Optional <i>ts</i> arguments</h2>By default, the <i>ts</i> command inserts the absolute timestamp into each output line. You can use the <i>-s</i> argument to replace the absolute timestamp with the elapsed duration since the start of execution. <br />
<code><br />
$ ansible-playbook -b -i hostsfile myPlaybook.yml |ts -s<br />
00:00:01 PLAY [localhost] ****************************************************************<br />
00:00:01 <br />
00:00:01 TASK [Gathering Facts] **********************************************************<br />
00:00:01 ok: [localhost]<br />
00:00:02 <br />
00:00:02 TASK [Disable Caps Lock] *********************************************************<br />
00:00:02 ok: [localhost]<br />
00:00:02 <br />
00:00:02 TASK [Install X apps] ***********************************************************<br />
00:00:06 changed: [localhost] => (item=autokey-gtk)<br />
00:00:10 changed: [localhost] => (item=gnucash)<br />
00:00:10 <br />
00:00:10 TASK [Install 64-bit texamker - Debian] *****************************************<br />
00:00:15 changed: [localhost]<br />
...snipped...<br />
00:00:26 PLAY RECAP **********************************************************************<br />
00:00:26 localhost : ok=8 changed=3 unreachable=0 failed=0 <br />
</code><br />
<code><br />
<br />
Another useful argument to know is <i>-i</i>. With this argument, each output line displays the elapsed time since the previous output line. You don't need to do the mental math to calculate how long a task took.<br />
<br />
$ ansible-playbook -b -i hostsfile myPlaybook.yml |ts -i<br />
00:00:00 PLAY [localhost] ****************************************************************<br />
00:00:00 <br />
00:00:00 TASK [Gathering Facts] **********************************************************<br />
00:00:01 ok: [localhost]<br />
00:00:00 <br />
00:00:00 TASK [Disable Caps Lock] *********************************************************<br />
00:00:00 ok: [localhost]<br />
00:00:00 <br />
00:00:00 TASK [Install X apps] ***********************************************************<br />
00:00:04 changed: [localhost] => (item=autokey-gtk)<br />
00:00:04 changed: [localhost] => (item=gnucash)<br />
00:00:00 <br />
00:00:00 TASK [Install 64-bit texamker - Debian] *****************************************<br />
00:00:05 changed: [localhost]<br />
...snipped...<br />
00:00:00 PLAY RECAP **********************************************************************<br />
00:00:00 localhost : ok=8 changed=3 unreachable=0 failed=0 <br />
</code><br />
<br />
In summary, <i>ts</i> is a fast and easy way to add timestamps to script or command output.<div class="blogger-post-footer"><script type="text/javascript"><!--
google_ad_client = "pub-1640070864477148";
//728x90, created 11/11/07
google_ad_slot = "6009857287";
google_ad_width = 728;
google_ad_height = 90;
//--></script>
<script type="text/javascript"
src="http://pagead2.googlesyndication.com/pagead/show_ads.js">
</script></div>Peter Leunghttp://www.blogger.com/profile/05589860210899238688noreply@blogger.com1tag:blogger.com,1999:blog-8032022811235182759.post-12845652552241932002018-05-01T10:51:00.000-07:002018-08-03T17:41:19.589-07:00Snaps as self-contained, auto-updating, universal software packages<div class="separator" style="clear: both; text-align: center;"><a href="https://2.bp.blogspot.com/-ekTNoI-vT4Q/WsUW30bGhtI/AAAAAAAACiE/R5JyEifD47AwGfBR5uva7tojHiGfV55IgCLcBGAs/s1600/snapcraftlogo.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="34" data-original-width="147" height="46" src="https://2.bp.blogspot.com/-ekTNoI-vT4Q/WsUW30bGhtI/AAAAAAAACiE/R5JyEifD47AwGfBR5uva7tojHiGfV55IgCLcBGAs/s200/snapcraftlogo.png" width="200" /></a></div>One of Linux's unique selling points is that users can choose from a variety of Linux distributions, each with its own features and advantages. However, a byproduct of the proliferation of distributions is that Linux developers are burdened with extra labor to package and deploy software in multiple incompatible package formats, such as RPM and DEB, using different package managers. <br />
<br />
<div id="line3">There have been several attempts to address this obstacle of deploying software across multiple Linux distributions. The latest such initiative, spearheaded by none other than the venerable Canonical Ltd, the Ubuntu developer, is <a href="https://snapcraft.io/">Snapcraft</a>. Snapcraft is the developer tool for packaging software in the universal snap format.</div><br />
<div id="line5">The rest of this post explains how to use snaps, from a user rather than developer's perspective.</div><br />
<font color=red>[2018-05-23 update]</font> A snap from the official Snap store <a href="https://blog.ubuntu.com/2018/05/15/trust-and-security-in-the-snap-store?m=1">was found</a> to contain hidden cryptocurrency mining code, and was since removed from the store. The news highlight the fact that the mere existence of a snap in the Snap store does not guarantee its integrity. The <a href="https://www.omgubuntu.co.uk/2018/05/ubuntu-snap-malware">lesson</a> is to only use snaps from trusted sources featured in the Snap store, such as the original software author, an official maintainer, or a trusted community source.<br />
<br />
<h2><a href="https://www.blogger.com/null" id="7"></a>What are snaps?</h2><div id="line8"><a href="https://en.wikipedia.org/wiki/Snappy_(package_manager)">Snaps</a> are universal software packages that can be deployed across the major Linux distributions and architectures including IoT. Although snaps is a format championed by the creator of Ubuntu, it is supported on all major Linux distributions including Debian, Ubuntu, Mint, Fedora, Gentoo, ArchLinux, Manjaro, and OpenSUSE.</div><br />
<div id="line10">Besides being distribution-agnostic, snaps are also self-contained. Linux users are all too familiar with "the dependency hell", ie, the occasional extreme frustration experienced in software installation due to dependency issues. A snap is self-contained in that it bundles the required runtime libraries inside the package.</div><br />
<div id="line12">Once you install a snap on your system, it will be auto-updated to run the latest release. You however can manually roll back to a previous version of the software if you so desire.</div><br />
<div id="line14">In summary, snaps are universal, self-contained, auto-updating Linux software packages.</div><br />
<h2><a href="https://www.blogger.com/null" id="16"></a>Install snapd</h2><div id="line17">To be able to run snap packages on your Linux system, you must first install <code>snapd</code>, the service responsible for running and managing snaps. The following command will install <code>snapd</code> on a Debian system.</div><pre><code>$ sudo apt install snapd
</code></pre><br />
<h2><a href="https://www.blogger.com/null" id="23"></a>The Snap store</h2><div id="line24">You can search the relatively small but growing online <a href="https://snapcraft.io/store">Snap store</a> for snaps to install. Alternatively, you can search using the command-line interface:</div><pre><code>$ snap find firefox
Name Version Developer Notes Summary
firefox 59.0.2-1 mozilla - Mozilla Firefox web browser
</code></pre><br />
<div id="line32">Why would you download the firefox snap when firefox is readily available as a DEB, via the Debian standard repository? Similarly, chromium and libreoffice belong to the same category of software. The answer is that, in most cases, snaps offer a much more recent version of the software than that from the native Linux package manager. For instance, Debian 9 ("Stretch") packages FireFox 52 in its standard repository whereas you can get Firefox 59 as a snap.</div><br />
The Snap store features some snaps that are not available from the standard repositories of major Linux distributions. Examples are ghostwriter and vidcutter. Instead of building such software manually, you can download and deploy their corresponding snaps from the Snap store.<br />
<br />
<div id="line36">Below is a non-exhaustive list of software available in snaps that I personally find useful(or fun).</div><ul id="line38"><li id="line38">Chromium</li>
<li id="line39">Firefox</li>
<li id="line39">Ghostwriter-casept (a Markdown editor) </li>
<li id="line40">LibreOffice</li>
<li id="line41">Minecraft</li>
<li id="line41">Nextcloud</li>
<li id="line41">OBS Studio (for screencasting and live video streaming)</li>
<li id="line41">Opera</li>
<li id="line42">Skype</li>
<li id="line43">Slack</li>
<li id="line44">Solitaire</li>
<li id="line45">Spotify</li>
<li id="line45">VidCutter (a video editor)</li>
</ul><br />
<div id="line47">To find out more about a particular snap, execute the following command:</div><pre><code>$ snap info minecraft
name: minecraft
summary: Minecraft is a game about placing blocks and going on adventures.
publisher: snapcrafters
license: Proprietary
description: A game about placing blocks while running from skeletons
snap-id: aJQRf6WPQq04DH0TB2HdTB6K9rf6I1yX
channels:
stable: latest (11) 148MB -
candidate: latest (11) 148MB -
beta: latest (11) 148MB -
edge: latest (11) 148MB -
</code></pre><br />
<h2><a href="https://www.blogger.com/null" id="64"></a>How to install a snap</h2><div id="line65">Installing a snap is as easy as:</div><pre><code>$ sudo snap install solitaire
</code></pre><br />
<div id="line70">If you don't have root privileges, you can still install snaps by first signing in to the Ubuntu Snap store. You will need an <a href="https://login.ubuntu.com/">Ubuntu One</a> account (which is free).</div><pre><code>$ sudo snap login <your-email-address>
$ snap install solitaire
</code></pre><div id="line77"><br />
To list the snaps already installed on your system:</div><pre><code>$ snap list
Name Version Rev Tracking Developer Notes
core 16-2.32.1 4327 stable canonical core
skype 8.18.0.6 23 stable skype classic
solitaire 1.0 2 stable 1bsyl devmode
</code></pre><br />
<div id="line86">To purge a snap from your local system:</div><pre><code>$ sudo snap remove solitaire
</code></pre><br />
<div id="line91">Note that the above command will delete all snap-specific data and settings.</div><br />
<h2><a href="https://www.blogger.com/null" id="93"></a>How to run a snap</h2><div id="line94">Installing a snap in most cases will automatically create a shortcut on your desktop menu system. For instance, you will find an entry for the solitaire snap in the <code>Games</code> sub-menu.</div><br />
<div id="line96">Alternatively, you can always run the snap command explicitly on the command line.</div><pre><code>$ snap run <your snap command>
</code></pre><br />
<div id="line101">The caveat with the command-line approach is that the snap argument may not always be obvious. For instance, with the solitaire snap, the corresponding name to use for the run command is <code>solitaire.1bsyl</code>, not <code>solitaire</code>. You can find out the specific name to use by examining snap's bin directory:</div><pre><code>$ ls /snap/bin/
skype solitaire.1bsyl
$ snap run solitaire.1bsyl
</code></pre><div class="blogger-post-footer"><script type="text/javascript"><!--
google_ad_client = "pub-1640070864477148";
//728x90, created 11/11/07
google_ad_slot = "6009857287";
google_ad_width = 728;
google_ad_height = 90;
//--></script>
<script type="text/javascript"
src="http://pagead2.googlesyndication.com/pagead/show_ads.js">
</script></div>Peter Leunghttp://www.blogger.com/profile/05589860210899238688noreply@blogger.com0tag:blogger.com,1999:blog-8032022811235182759.post-13905940486672588332018-04-03T14:32:00.000-07:002018-04-03T14:32:00.716-07:00Scanning HTTPS for Mixed Content<div class="separator" style="clear: both; text-align: center;"><a href="https://3.bp.blogspot.com/-VShdRDcM-Dw/Woc5IMqQ3AI/AAAAAAAACgo/v7mvIR0EOGsi3mud0thvynpirTivkwkVwCLcBGAs/s1600/digital-579553_1920.jpg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="1131" data-original-width="1600" height="226" src="https://3.bp.blogspot.com/-VShdRDcM-Dw/Woc5IMqQ3AI/AAAAAAAACgo/v7mvIR0EOGsi3mud0thvynpirTivkwkVwCLcBGAs/s320/digital-579553_1920.jpg" width="320" /></a></div><div id="line2">Back in 2014, Google raised the awareness of using <a href="https://en.wikipedia.org/wiki/HTTPS">HTTPS</a> ("Secure HTTP") by <a href="https://webmasters.googleblog.com/2014/08/https-as-ranking-signal.html">making its use a ranking signal in Google search algorithms</a>. HTTPS essentially establishes secure encrypted connections to the cloud. Google further raised the stake of not using HTTPS by <a href="https://blog.chromium.org/2018/02/a-secure-web-is-here-to-stay.html">announcing that</a>, beginning in July 2018, the Google Chrome browser with the release of Chrome 68 will mark <strong>all</strong> HTTP websites as being insecure. The consequence of not converting to HTTPS is that site visitors will be persuaded by the warning message to bounce from your website.</div><div class="separator" style="clear: both; text-align: center;"><a href="https://2.bp.blogspot.com/-LaXBt1h-Rvo/WsKO9w9FKYI/AAAAAAAAChU/yQFsCyP4LusS7ULXQoCizKN_hKltEHqwACLcBGAs/s1600/FireFoxInsecureWarning.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="226" data-original-width="516" height="140" src="https://2.bp.blogspot.com/-LaXBt1h-Rvo/WsKO9w9FKYI/AAAAAAAAChU/yQFsCyP4LusS7ULXQoCizKN_hKltEHqwACLcBGAs/s320/FireFoxInsecureWarning.png" width="320" /></a></div>Even before the impending drop dead date, Chrome and other popular web browsers such as Firefox and Edge have been warning visitors to HTTP-connected sites with an informational message. <br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;"><a href="https://4.bp.blogspot.com/-ix9PaRXLuCQ/WsK4C7j8ZnI/AAAAAAAAChk/5RfjZtJLVsUG2qul0-qleGi_MNZ_4xhWwCLcBGAs/s1600/FirefoxSecure.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="230" data-original-width="514" height="143" src="https://4.bp.blogspot.com/-ix9PaRXLuCQ/WsK4C7j8ZnI/AAAAAAAAChk/5RfjZtJLVsUG2qul0-qleGi_MNZ_4xhWwCLcBGAs/s320/FirefoxSecure.png" width="320" /></a></div><div id="line9">Web administrators had taken heed and converted their websites to HTTPS, many taking advantage of the free SSL certificates issued by <a href="https://letsencrypt.org/">Let's Encrypt</a>. However, if you have successfully converted to HTTPS, your work may not be done. You still need to verify that your website is properly recognized as being secure. You want to see the padlock icon displayed next to the web page's URL in the browser window.</div><br />
<div id="line11">To many administrators' surprise, even a properly converted HTTPS website may still be marked as being insecure. This is most likely due to the website's <a href="https://developers.google.com/web/fundamentals/security/prevent-mixed-content/what-is-mixed-content">mixed content</a>. For a web page to be deemed secure, everything loaded by that page must be encrypted by HTTPS. A web page with mixed content loads both encrypted as well as non-encrypted contents such as images, videos, stylesheets and scripts.</div><br />
<div id="line13">While it is possible to manually spot mixed web content on a web page, checking a non-trivial website requires automation. <a href="https://github.com/bramus/mixed-content-scan">Mixed Content Scan</a> is a command-line web crawler which scans for mixed content. The rest of this post explains how to install and use the tool.</div><br />
<h2><a href="https://www.blogger.com/null" id="15"></a>Installation</h2><div id="line16"><code>Mixed Content Scan</code> is a batch PHP application. To install the tool, use <code>composer</code>, a PHP package dependency manager. For the latest instructions on how to install <code>composer</code>, please refer to <a href="https://getcomposer.org/download/">this link</a>. Note that the said procedure installs <code>composer</code> in the current directory. Optionally, move the executable to a globally accessible directory using the following command.</div><pre><code>$ sudo mv composer.phar /usr/local/bin/composer
</code></pre><div id="line21">To install <code>Mixed Content Scan</code>:</div><pre><code>$ composer global require bramus/mixed-content-scan:~2.8
</code></pre><div id="line25">The <code>Mixed Content Scan</code> executable is placed in <code>~/.config/composer/vendor/bramus/mixed-content-scan/bin</code>.</div><br />
<h2><a href="https://www.blogger.com/null" id="27"></a>Scanning for mixed content</h2><div id="line29">To scan a website for mixed content, simply provide its URL as an argument to <code>Mixed Content Scan</code>:</div><pre><code>$ cd ~/.config/composer/vendor/bramus/mixed-content-scan/bin
$ ./mixed-content-scan https://shadowofyourwings.com/
</code></pre><div id="line35">By default, the tool outputs the scan report on the terminal("standard output"). Alternatively, you can specify an output file using the <code>--output</code> parameter as follows:</div><pre><code>$ cd ~/.config/composer/vendor/bramus/mixed-content-scan/bin
$ ./mixed-content-scan --output <some/file/path> https://shadowofyourwings.com/
</code></pre><div id="line42">You can also use the <code>--ignore</code> parameter to specify a file which contains URL patterns that the tool will ignore and not scan. The example site I use is a WordPress website. The scanning tool comes with a sample <code>ignore</code> file for WordPress which is located in <code>~/.config/composer/vendor/bramus/mixed-content-scan/bin/ignorepatterns/wordpress.txt</code>.</div><div id="line44"><code><br />
<br />
$ cd ~/.config/composer/vendor/bramus/mixed-content-scan/bin<br />
$ ./mixed-content-scan --ignore=~/.config/composer/vendor/bramus/mixed-content-scan/bin/ignorepatterns/wordpress.txt https://shadowofyourwings.com/<br />
[2018-02-16 16:53:18] MCS.NOTICE: Scanning https://shadowofyourwings.com/<br />
[2018-02-16 16:53:18] MCS.ERROR: 00000 - https://shadowofyourwings.com/<br />
[2018-02-16 16:53:18] MCS.WARNING: http://gmpg.org/xfn/11<br />
<span style="color: red;">[2018-02-16 16:53:19] MCS.ERROR: 00001 - https://shadowofyourwings.com/about<br />
[2018-02-16 16:53:19] MCS.WARNING: http://shadowofyourwings.com/wp-content/uploads/2017/05/peterLeung.jpg<br />
[2018-02-16 16:53:19] MCS.WARNING: http://gmpg.org/xfn/11</span><br />
[2018-02-16 16:53:20] MCS.ERROR: 00002 - https://shadowofyourwings.com/contact<br />
[2018-02-16 16:53:20] MCS.WARNING: http://gmpg.org/xfn/11<br />
... <output snipped> ...<br />
[2018-02-16 16:53:38] MCS.NOTICE: Scanned 26 pages for Mixed Content<br />
</code></div><br />
<div id="line59"><code>Mixed Content Scan</code> numbers each page scanned, starting from <code>00000</code>. In the above example, the <code>About</code> page (<code>00001</code>) has been flagged as having mixed content. The sources of mixed content as loaded by that page are twofold:</div><ol id="line61"><li id="line61"><div id="line61">Vulnerable image file.<br />
The <code>peterLeung.jpg</code> file is being loaded via the insecure HTTP connection. The fix is simple: go to the WordPress administration web page, and change HTTP to HTTPS on the <code>About</code> web page.</div></li>
<li id="line63"><div id="line63">Theme header profile<br />
The header of the default twentyseventeen WordPress theme contains a reference to <a href="http://gmpg.org/xfn/11">http://gmpg.org/xfn/11</a>. The code is in <code><document root>/wp-content/themes/twentyseventeen/header.php</code>.<br />
<br />
Although the scanner reports its occurrence as a violation, browsers generally do not flag this as a mixed content error. This error can be safely ignored.</div></li>
</ol><div class="blogger-post-footer"><script type="text/javascript"><!--
google_ad_client = "pub-1640070864477148";
//728x90, created 11/11/07
google_ad_slot = "6009857287";
google_ad_width = 728;
google_ad_height = 90;
//--></script>
<script type="text/javascript"
src="http://pagead2.googlesyndication.com/pagead/show_ads.js">
</script></div>Peter Leunghttp://www.blogger.com/profile/05589860210899238688noreply@blogger.com1tag:blogger.com,1999:blog-8032022811235182759.post-91496998351037023862018-03-16T10:25:00.001-07:002018-03-16T10:25:47.701-07:00A review of 3 best-of-breed Markdown editors<p><a href="https://3.bp.blogspot.com/-MwaZVjmiuvQ/WoS_SNv4lQI/AAAAAAAACe4/yOg0ABcfFsoDXcTvpy2tp5W374x_O2KRQCLcBGAs/s1600/html-2769364_1920.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="1200" data-original-width="1600" height="240" src="https://3.bp.blogspot.com/-MwaZVjmiuvQ/WoS_SNv4lQI/AAAAAAAACe4/yOg0ABcfFsoDXcTvpy2tp5W374x_O2KRQCLcBGAs/s320/html-2769364_1920.png" width="320" /></a><br />As a technology blogger, I write HTML documents that are hosted on different platforms such as WordPress, Drupal, and Blogger. I like to compose HTML using the <a href="https://en.wikipedia.org/wiki/Markdown">Markdown markup language</a>. Unfortunately, the HTML editors bundled with the aforementioned platforms do not support Markdown natively. It is true that you can download Markdown plugins for WordPress and Drupal. But, at the end, I still find the HTML editors to be too intrusive for a writer such as myself to stay focused and productive.</p>
<p>Fortunately, there are many good special-purpose Markdown editors out there. My web authoring process involves first composing the document using a Markdown editor, and then copying and pasting the output HTML into the Content Management System(CMS). Below, I evaluate 3 open-source Markdown editors: justmd, Remarkable, and ghostwriter. </p>
<p>I will evaluate each editor from two sometimes conflicting viewpoints, that of a geek and a writer. As a geek, I side with editors that have many bells and whistles. But, as a writer, I prefer editors that help me create, often by filtering out as much distraction as possible, and forcing me to focus on the next word, phrase, sentence to put on the page.</p>
<h2>justmd</h2>
<p><a href="https://4.bp.blogspot.com/-jfNfBsjLRWk/WoS0TqhLqhI/AAAAAAAACec/8E3s9_mx_FoGJt7qfJdhaE1vbG-bgssYgCLcBGAs/s1600/justmdwindow.png" imageanchor="1" ><img border="0" src="https://4.bp.blogspot.com/-jfNfBsjLRWk/WoS0TqhLqhI/AAAAAAAACec/8E3s9_mx_FoGJt7qfJdhaE1vbG-bgssYgCLcBGAs/s320/justmdwindow.png" width="320" height="169" data-original-width="1600" data-original-height="845" /></a></p>
<p><a href="http://www.i38.me/justmd/">justmd</a> is a minimalist, bare-bones Markdown editor. When you open justmd, you will see a single window with 2 window panes of equal size, located side-by-side. One pane is where you enter the Markdown text; the other is the HTML preview pane. Although you can change the overall size of the encompassing window, you cannot change the ratio of the 2 panes. The geek in me cannot help but cringe at the discovery. After all, it is common among Markdown editors (including Remarkable and ghostwriter) to have separate input and preview windows that you can independently resize and even hide. Conversely, the writer in me gives justmd a big shout-out for its austere simplicity. You just open the app, and immediately start writing, without having to adjust the size of any window component. Writers will find justmd more conducive to writing than many editors that are much more customizable.</p>
<p><a href="https://2.bp.blogspot.com/-yjqrvrFWr-k/WoS93JVQGkI/AAAAAAAACes/t44Quf1AxZkwk8OFPbxUo1qnkMFBgE_BgCLcBGAs/s1600/pencil-3150319_1920.jpg" imageanchor="1" ><img border="0" src="https://2.bp.blogspot.com/-yjqrvrFWr-k/WoS93JVQGkI/AAAAAAAACes/t44Quf1AxZkwk8OFPbxUo1qnkMFBgE_BgCLcBGAs/s320/pencil-3150319_1920.jpg" width="320" height="187" data-original-width="1600" data-original-height="937" /></a></p>
<p>Minimalist as it is designed to be, justmd, as a Markdown editor, is not feature complete in its current status. The following features, which I deem to be very important for writers, are still missing in justmd: </p>
<ul>
<li>Spellchecker.</li>
<li>Word count.</li>
<li>Auto save.</li>
</ul>
<p>This post was written entirely using justmd, and the overall experience was very positive. The lack of a spellchecker and word counter did not hamper the writing at all. On the contrary, it enhances my productivity by breaking the bad habits of constantly checking the word count and looking out for spelling errors in the midst of writing. Most Content Management Systems are capable of spellchecking and word counting. So, those tasks can be deferred until later, after you paste the HTML into the CMS. </p>
<p>Finally, I comment on the ease of installing justmd. None of the 3 editors being reviewed here are pre-packaged in the official repository of a major Linux distribution. Having said that, installing justmd is as easy as 1-2-3.</p>
<ol>
<li>Download compressed tarball from <a href="http://www.i38.me/justmd/">justmd website</a>.</li>
<li>Uncompress the tarball using command <code>tar -zxvf justmd-linux-x64-v1.1.1.tar.gz</code>.</li>
<li>Create shortcut to justmd binary.</li>
</ol>
<h2>Remarkable</h2>
<p><a href="https://3.bp.blogspot.com/-sMY3tEKApaM/WoTnIdzVQyI/AAAAAAAACgA/9WcFT24UDWEvimR-1_ESaL9aamD8K5I9gCLcBGAs/s1600/remarkablescreen.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="845" data-original-width="1600" height="169" src="https://3.bp.blogspot.com/-sMY3tEKApaM/WoTnIdzVQyI/AAAAAAAACgA/9WcFT24UDWEvimR-1_ESaL9aamD8K5I9gCLcBGAs/s320/remarkablescreen.png" width="320" /></a></p>
<p>Featurewise, <a href="http://remarkableapp.github.io/">Remarkable</a> is middle-of-the-road, between justmd and ghostwriter. It has word counting, but no spellchecking. Like justmd, both input and preview functions coexist as panes side-by-side in a single window, but you can stack them vertically or horizontally, and you can resize each pane proportionally within the window.</p>
<p>Now, as a writer, I find Remarkable's user interface too colorful, too distracting. Specifically, its overly generous use of color for syntax highlighting and icon design is detrimental to the primary writing task. With color, less is more.</p>
<p>You can download the Remarkable package in <code>.deb</code> or <code>.rpm</code> format from its <a href="https://remarkableapp.github.io/linux/download.html">Linux download page</a>. Users of Debian, Ubuntu, Fedora, SUSE, and Arch systems will find installation straightforward.</p>
<h2>ghostwriter</h2>
<div class="separator" style="clear: both; text-align: center;"><a href="https://4.bp.blogspot.com/-s83cxrlOgr8/WoTOzZTFEAI/AAAAAAAACfg/4bb5CN1A-nkqloE4yqE2TNvkos79O_-tQCLcBGAs/s1600/ghostwriter.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://4.bp.blogspot.com/-s83cxrlOgr8/WoTOzZTFEAI/AAAAAAAACfg/4bb5CN1A-nkqloE4yqE2TNvkos79O_-tQCLcBGAs/s1600/ghostwriter.png" data-original-width="128" data-original-height="128" /></a></div>
<p><a href="https://wereturtle.github.io/ghostwriter/">ghostwriter</a> is the most mature and feature complete of all 3 MarkDown editors. It offers spellchecking, word counting, auto saves, and much more.</p>
<p>Two unique features are especially noteworthy to writers: Hemingway and Focus modes. In Hemingway mode, two particular keyboard keys are disabled, namely, the <code>delete</code> and the <code>backspace</code> keys. The rationale is to increase productivity by delaying document editing as much as possible. In Focus mode, only the portion of the document you are working on is made prominent, and the rest fades out. You can configure the focus to be the current sentence, the current single or 3 lines, or the current paragraph. </p>
<div class="separator" style="clear: both;"><a href="https://3.bp.blogspot.com/-rbuKg7b1cgw/WoTTpoiZKKI/AAAAAAAACfw/s7MbOrhRFB4mgrjBZuRz2WjQh1RkecJLgCLcBGAs/s1600/ghostwriterscreen.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://3.bp.blogspot.com/-rbuKg7b1cgw/WoTTpoiZKKI/AAAAAAAACfw/s7MbOrhRFB4mgrjBZuRz2WjQh1RkecJLgCLcBGAs/s320/ghostwriterscreen.png" width="320" height="153" data-original-width="1600" data-original-height="767" /></a></div>
<p>Despite the rich feature set, the ghostwriter user interface is surprising clean and uncluttered. </p>
<p>The input and live preview functions reside in separate windows that you can resize and move around individually. Keen observers will definitely notice there is a real-time lag between actual text input and the update of the live preview. This is not a bug in the program. On the contrary, ghostwriter is programmed to only update the live preview when you stop typing(for a fraction of a second). The technical reason given by the developers is that the delay smoothens the jitters in synchronizing the rendering of large files. I can see many writers actually support this design decision because attention should be primarily focused on the writing, not the rendering, of the document.</p>
<p>Recall that the overall objective for using a MarkDown editor is to generate HTML code to insert into a CMS. With justmd and Remarkable, you need to first export to a HTML file, and then import the file (or copy and paste its contents) into the CMS. On the other hand, ghostwriter provides a shortcut <code>Copy HTML</code> button which is discreetly tucked away at the bottom right of the window. The button is a minor feature in the overall design scheme, but has a disproportionally high value to end users. Clicking the button copies the HTML code in its entirety into the clipboard. Importing the HTML into the CMS simply involves pasting the contents of the clipboard.</p>
<p>ghostwriter provides packages for Ubuntu , Fedora, openSUSE, and Arch Linux AUR. If you run Ubuntu or any of its derivatives such as Linux Mint, ghostwriter can be installed after adding a PPA repository and updating the local cache.</p>
<pre><code>sudo add-apt-repository ppa:wereturtle/ppa
sudo apt update
sudo apt install ghostwriter
</code></pre>
<p>If ghostwriter is not pre-packaged for your distro, e.g., Debian, you can follow the <a href="https://github.com/wereturtle/ghostwriter/blob/master/README.md">on-line instructions</a> to build the executable yourself. Depending on the particular distro and release, be prepared to spend some considerable time as you may run into the proverbial Linux dependency hell. </p>
<h2>Feature comparison</h2>
<table border="1"><thead>
<tr>
<th>Features</th>
<th>justmd</th>
<th>Remarkable</th>
<th>ghostwriter</th>
</tr>
</thead><tbody>
<tr>
<td>Cross-platform</td>
<td>Linux(x64), Windows(x64), macOS</td>
<td>Linux, Windows</td>
<td>Linux, Windows</td>
</tr>
<tr>
<td>Linux installation</td>
<td>Downloadable executables</td>
<td>Downloadble packages for Debian, Ubuntu, Fedora, openSUSE, Arch</td>
<td>Downloadble packages for Ubuntu, Fedora, openSUSE, Arch</td>
</tr>
<tr>
<td>Export to HTML, PDF</td>
<td>Yes</td>
<td>Yes</td>
<td>HTML, PDF, Word, ODT</td>
</tr>
<tr>
<td>Spellchecker</td>
<td>No</td>
<td>No</td>
<td>Yes</td>
</tr>
<tr>
<td>Auto save</td>
<td>No</td>
<td>No</td>
<td>Yes</td>
</tr>
<tr>
<td>Word count</td>
<td>No</td>
<td>Character, word, line counts</td>
<td>Character, word, line, sentence, paragraph, page counts</td>
</tr>
<tr>
<td>Live preview</td>
<td>Fixed window proportion</td>
<td>Hidable, variable proportion</td>
<td>Separate resizable window (no dual panel)</td>
</tr>
<tr>
<td><a href="https://guides.github.com/features/mastering-markdown/#GitHub-flavored-markdown">GitHub-flavored syntax</a></td>
<td>Support for tables</td>
<td>Yes (tables, strikethrough, emphasis, etc)</td>
<td>Yes (tables, strikethrough, emphasis, etc)</td>
</tr>
</tbody></table>
<h2>Summary & conclusion</h2>
<p><a href="https://4.bp.blogspot.com/-HhuVq1rGDy8/WoZduK4QXCI/AAAAAAAACgU/Xqz9-atnJKs5LTZINf8pShVBeYfgvFwaQCLcBGAs/s1600/scale-2635397_1280.jpg" imageanchor="1" ><img border="0" src="https://4.bp.blogspot.com/-HhuVq1rGDy8/WoZduK4QXCI/AAAAAAAACgU/Xqz9-atnJKs5LTZINf8pShVBeYfgvFwaQCLcBGAs/s200/scale-2635397_1280.jpg" width="200" height="111" data-original-width="1280" data-original-height="711" /></a>A writer's working style is intrinsically idiosyncratic. A writing environment that is distraction-free to one person may not be stimulating enough for another. Yet, ghostwriter is the clear winner of the 3 editors because it strikes a balance between clean design and feature richness. However, if ghostwriter is not pre-packaged for your Linux distro (say Debian), justmd and Remarkable are definitely worthwhile alternatives. </p>
<div class="blogger-post-footer"><script type="text/javascript"><!--
google_ad_client = "pub-1640070864477148";
//728x90, created 11/11/07
google_ad_slot = "6009857287";
google_ad_width = 728;
google_ad_height = 90;
//--></script>
<script type="text/javascript"
src="http://pagead2.googlesyndication.com/pagead/show_ads.js">
</script></div>Peter Leunghttp://www.blogger.com/profile/05589860210899238688noreply@blogger.com2tag:blogger.com,1999:blog-8032022811235182759.post-34575679430756750712018-03-01T04:30:00.000-08:002018-03-01T09:34:06.218-08:00Sharing folders between VirtualBox host and guest machines<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-j9AdICaQPeU/WoM06kLqjcI/AAAAAAAACdc/ElYc4joaXDcgjXLEH06-8Kn68Dk38vDSwCLcBGAs/s1600/Virtualbox_logo.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="512" data-original-width="512" height="320" src="https://1.bp.blogspot.com/-j9AdICaQPeU/WoM06kLqjcI/AAAAAAAACdc/ElYc4joaXDcgjXLEH06-8Kn68Dk38vDSwCLcBGAs/s320/Virtualbox_logo.png" width="320" /></a></div>
<br />
This post is the 4th installment of the VirtualBox series. The focus is on how to share folders between the host and the guest OSes. <a href="https://linuxcommando.blogspot.com/2017/12/using-virtualbox-to-build-freebsd-vm-on.html">Part 1</a> of the series shows how to install VirtualBox on a Debian host, and how to create a FreeBSD virtual machine. <a href="https://linuxcommando.blogspot.com/2018/01/what-to-do-after-installing-virtualbox.html">Part 2</a> outlines the post installation tasks, including installing the universal VirtualBox extension pack. <a href="https://linuxcommando.blogspot.com/2018/02/how-to-mount-usb-drives-on-virtualbox.html">Part 3</a> shows how the guest OS can access USB drives mounted on the host.<br />
The method outlined in this blog post works for Linux and Windows guest OSes but not FreeBSD. You can set up NFS or Samba to share a folder between a Linux host and a FreeBSD guest.<br />
<br />
<h2>
Prerequisites</h2>
To share a folder using the following procedure, the OS-specific guest additions package must be pre-installed. <a href="https://linuxcommando.blogspot.com/2017/12/using-virtualbox-to-build-freebsd-vm-on.html">Part 1</a> of this series includes instructions for installing the guest additions package for the FreeBSD guest OS.<br /><br />
<h2>
Procedure</h2>
<ol>
<li>Declare the shared folder on host OS.<br />
Open the <code>Oracle VM VirtualBox manager</code> on your host machine, select the target guest OS, and click <code>Settings</code>. Note that your guest machine does NOT need to be powered off.<br />
<a href="https://4.bp.blogspot.com/-KWIUXqDZp2Q/WoM6eMYGfuI/AAAAAAAACds/S0uRAp0jKzIqOhl5ly_X7cf0xm2feRF4wCLcBGAs/s1600/VboxSettings.png" imageanchor="1"><img border="0" data-original-height="680" data-original-width="1042" height="209" src="https://4.bp.blogspot.com/-KWIUXqDZp2Q/WoM6eMYGfuI/AAAAAAAACds/S0uRAp0jKzIqOhl5ly_X7cf0xm2feRF4wCLcBGAs/s320/VboxSettings.png" width="320" /></a></li>
<li>Select <code>Shared Holders</code>, and click the <code>Plus</code> button to add a shared folder. Note that you can have more than 1 shared folder.<br />
<a href="https://3.bp.blogspot.com/-MU4VT6FyJt0/WoNAc5vE9TI/AAAAAAAACd8/1ThYs_d3YboFaKfKlAPERiaTP1sfIyOSwCLcBGAs/s1600/SharedFolders.png" imageanchor="1"><img border="0" data-original-height="456" data-original-width="706" height="207" src="https://3.bp.blogspot.com/-MU4VT6FyJt0/WoNAc5vE9TI/AAAAAAAACd8/1ThYs_d3YboFaKfKlAPERiaTP1sfIyOSwCLcBGAs/s320/SharedFolders.png" width="320" /></a></li>
<li>Specify the path for the shared folder.<br />
Click the <code>down</code> arrow next to the <code>Folder Path</code> parameter, select <code>Other,</code> and navigate to the target shared folder.<br />
<a href="https://3.bp.blogspot.com/-lLQ8vYYUM4I/WoNbYmvkI7I/AAAAAAAACeM/8ENzyf3VsR8E2R7h56Vg1_3BNYaJ9H-7ACLcBGAs/s1600/AddSharePath.png" imageanchor="1"><img border="0" data-original-height="257" data-original-width="320" height="257" src="https://3.bp.blogspot.com/-lLQ8vYYUM4I/WoNbYmvkI7I/AAAAAAAACeM/8ENzyf3VsR8E2R7h56Vg1_3BNYaJ9H-7ACLcBGAs/s320/AddSharePath.png" width="320" /></a></li>
<li>Configure the shared folder.<br />
The <code>Folder name</code> field refers to the folder name on the guest OS, which defaults to the folder name you specified in the last step. You should note the folder name in order to identify the full folder pathname on the guest OS. For example, if the shared folder on the host machine is <code>/home/peter/Music</code>, the corresponding folder name defaults to <code>/media/sf_Music</code> on the Ubuntu guest OS, and <code>\\vboxsvr\Music</code> on the Windows guest OS.<br /><br />
The <code>Auto-mount</code> checkbox should be enabled. Optionally, you can also mount the folder manually, but then you must modify access permissions in order to make the folder writable by non-root users.<br /><br />
Unless the shared folder is for one-off use only, you should enable the <code>Make Permanent</code> checkbox. </li>
<li>Power recycle the guest machine. </li>
<li>Grant folder access permission to <code>non-root</code> users.<br />
This step only applies to Linux guest OS(not Windows). Login to the guest OS, and add regular users to the <code>vboxsf</code> group using the following command.<br /><br />
<code>sudo usermod -aG vboxsf <someuserID></code></li>
</ol><br />
<h2>
Related posts</h2>
<ul>
<li><a href="https://linuxcommando.blogspot.com/2017/12/using-virtualbox-to-build-freebsd-vm-on.html">Part 1 of VirtualBox series - on installation</a>.</li>
<li><a href="https://linuxcommando.blogspot.com/2018/01/what-to-do-after-installing-virtualbox.html">Part 2 - post-installation</a>.</li>
<li><a href="https://linuxcommando.blogspot.com/2018/02/how-to-mount-usb-drives-on-virtualbox.html">Part 3 - mounting of USB drives</a>.</li>
</ul>
<div class="blogger-post-footer"><script type="text/javascript"><!--
google_ad_client = "pub-1640070864477148";
//728x90, created 11/11/07
google_ad_slot = "6009857287";
google_ad_width = 728;
google_ad_height = 90;
//--></script>
<script type="text/javascript"
src="http://pagead2.googlesyndication.com/pagead/show_ads.js">
</script></div>Peter Leunghttp://www.blogger.com/profile/05589860210899238688noreply@blogger.com0tag:blogger.com,1999:blog-8032022811235182759.post-46454101183086669552018-02-14T05:00:00.000-08:002018-02-14T05:00:16.977-08:00Ubuntu: how to reset lost administrative password<div class="separator" style="clear: both; text-align: center;">
<a href="https://4.bp.blogspot.com/-e0Thgpa_ReI/Wk6_IwJsrBI/AAAAAAAACY0/pa0uP1PRhM80uvAVcBMl7up42jVUup_8wCLcBGAs/s1600/panic-1393619_640.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="384" data-original-width="640" height="192" src="https://4.bp.blogspot.com/-e0Thgpa_ReI/Wk6_IwJsrBI/AAAAAAAACY0/pa0uP1PRhM80uvAVcBMl7up42jVUup_8wCLcBGAs/s320/panic-1393619_640.png" width="320" /></a></div>
After an extended vacation, I came home to discover to my horror that I could no longer login to my seldom-used Ubuntu laptop. The reason was embarrassingly simple: I forgot my password. My muscle memory (or finger memory) did not help while I was frantically typing my usual passwords but to no avail. I own the administrative account on that system. So, I had no one else to turn to for help. Luckily, I was able to login another machine and google how to reset the administrative password on Ubuntu.
<br />
<h2>
<a href="https://www.blogger.com/null" id="4"></a>Prerequisites</h2>
<div id="line5">
There are 2 preconditions for using the procedure to reset the administrative password.</div>
<ol id="line7">
<li id="line7">
<div id="line7">
Physical access to machine.<br />
You need to access the system console in order to interrupt loading of the OS.</div>
</li>
<li id="line9">
<div id="line9">
The <code>root</code> password was disabled.<br />
By default, Ubuntu disables the password of the <code>root</code> account by assigning it a value which cannot possibly match any encrypted value. During the installation of Ubuntu, you were asked to create the first user. That initial user, by default, belongs to the <code>sudo</code> group which means that the user can be elevated to perform system administration functions. The administrative password which we are going to reset refers to the password of the initial user.</div>
<div id="line12">
The procedure assumes that you did not manually assign <code>root</code> a valid password. If <code>root</code> has a valid password and you know it, then you can simply sign on as <code>root</code> and reset the administrative password using the <code>password</code> command. If the <code>root</code> password was also forgotten, this procedure does not apply because you need to enter that password to drop to <code>root</code> shell prompt in this procedure.</div>
</li>
</ol>
<h2>
<a href="https://www.blogger.com/null" id="14"></a>How to reset administrative password</h2>
<ol id="line15">
<li id="line15">
<div id="line15">
Power recycle.<br />
After the BIOS screen appears, press down the <code>left shift</code> key to enter <code>GRUB</code>. This step can be quite finicky, and you may need to repeat it several times until you get the timing just right.</div>
<div id="line18">
<a href="https://2.bp.blogspot.com/-zPNJ522sH1E/WlPTZnyD_vI/AAAAAAAACZU/HIYRw7WAoQAFAozkAVWUZDPUk64Fx277ACLcBGAs/s1600/grub.png" imageanchor="1"><img border="0" data-original-height="480" data-original-width="640" height="300" src="https://2.bp.blogspot.com/-zPNJ522sH1E/WlPTZnyD_vI/AAAAAAAACZU/HIYRw7WAoQAFAozkAVWUZDPUk64Fx277ACLcBGAs/s400/grub.png" width="400" /></a></div>
</li>
<li id="line20">
<div id="line20">
Scroll down to <code>Advanced options for Ubuntu</code>, and press <code>Enter</code>.</div>
<div id="line22">
<a href="https://1.bp.blogspot.com/-XmatdxaaDDY/WlPUoIZU3ZI/AAAAAAAACZY/WoWaLogUSZkpKciSo6gvOd_iXxMSqBa2ACLcBGAs/s1600/recoverymode.png" imageanchor="1"><img border="0" data-original-height="480" data-original-width="640" height="300" src="https://1.bp.blogspot.com/-XmatdxaaDDY/WlPUoIZU3ZI/AAAAAAAACZY/WoWaLogUSZkpKciSo6gvOd_iXxMSqBa2ACLcBGAs/s400/recoverymode.png" width="400" /></a></div>
</li>
<li id="line24">
<div id="line24">
In the ensuing screen, scroll down to the top <code>Recovery mode</code> line, and press <code>Enter</code>.<br />
If the Linux kernel image had been upgraded on the machine before, you would see multiple <code>recover mode</code> lines on the screen. Select the <code>recovery mode</code> line that corresponds to the latest Linux image(that is nearest to the top).</div>
<div id="line27">
<a href="https://2.bp.blogspot.com/-SwzXzQa-J5g/WlPWKPFzCxI/AAAAAAAACZc/Ub_lBahlMLsSSoLlz6Tb7sqGNS15KEr8QCLcBGAs/s1600/recoverymenu.png" imageanchor="1"><img border="0" data-original-height="400" data-original-width="720" height="222" src="https://2.bp.blogspot.com/-SwzXzQa-J5g/WlPWKPFzCxI/AAAAAAAACZc/Ub_lBahlMLsSSoLlz6Tb7sqGNS15KEr8QCLcBGAs/s400/recoverymenu.png" width="400" /></a></div>
</li>
<li id="line29">
<div id="line29">
In the <code>Recovery Menu</code> screen, scroll down to <code>root</code>, and press <code>Enter</code>.</div>
<div id="line31">
<a href="https://3.bp.blogspot.com/-nfccUFgmiqs/WlPXBI1MpyI/AAAAAAAACZg/tLBj3s2wgeIaiQZFhRSidEPmPXrKhCKQgCLcBGAs/s1600/maintenance.png" imageanchor="1"><img border="0" data-original-height="400" data-original-width="720" height="222" src="https://3.bp.blogspot.com/-nfccUFgmiqs/WlPXBI1MpyI/AAAAAAAACZg/tLBj3s2wgeIaiQZFhRSidEPmPXrKhCKQgCLcBGAs/s400/maintenance.png" width="400" /></a></div>
</li>
<li id="line33">
<div id="line33">
Press <code>Enter</code> again.<br />
If you have previously assigned <code>root</code> a password, you would be prompted to enter it at this step. Otherwise, just press <code>Enter</code> to continue.</div>
<div id="line36">
<a href="https://2.bp.blogspot.com/-xilBb1UacUM/WlPXtg1MWLI/AAAAAAAACZk/evWmCEcD1pQJ23Gfvk25FVy_7SwProMwwCLcBGAs/s1600/rootprompt.png" imageanchor="1"><img border="0" data-original-height="400" data-original-width="720" height="222" src="https://2.bp.blogspot.com/-xilBb1UacUM/WlPXtg1MWLI/AAAAAAAACZk/evWmCEcD1pQJ23Gfvk25FVy_7SwProMwwCLcBGAs/s400/rootprompt.png" width="400" /></a></div>
</li>
<li id="line38">
<div id="line38">
Remount filesystem.<br />
After all the hard work, you are now at the <code>root shell prompt</code>. The filesystem at this point is read-only. Remount the file system to add write permission.</div>
<pre><code># mount -o rw,remount /
</code></pre>
</li>
<li id="line44">
<div id="line44">
Reset administrative password.<br />
Use the <code>passwd</code> command to change the password for the administrative user.</div>
<pre><code># passwd <adminuser>
</code></pre>
<div id="line51">
Press <code>Control-D</code> to return to the <code>recovery menu</code>.</div>
<div id="line53">
<a href="https://2.bp.blogspot.com/-tE1xo24U4XY/WlPYiDbERAI/AAAAAAAACZo/fWaoqG_nvE4sr71h5xE4b0ayr8tDmQqlACLcBGAs/s1600/resume.png" imageanchor="1"><img border="0" data-original-height="400" data-original-width="720" height="222" src="https://2.bp.blogspot.com/-tE1xo24U4XY/WlPYiDbERAI/AAAAAAAACZo/fWaoqG_nvE4sr71h5xE4b0ayr8tDmQqlACLcBGAs/s400/resume.png" width="400" /></a></div>
</li>
<li id="line55">
<div id="line55">
Select resume.</div>
<div id="line57">
<a href="https://1.bp.blogspot.com/-mDZD4P5yKyQ/WlPY-7f5kMI/AAAAAAAACZs/1K9HGOfUn0c1GR6Os5yXY5ygkd48YGgswCLcBGAs/s1600/informationscreen.png" imageanchor="1"><img border="0" data-original-height="400" data-original-width="720" height="222" src="https://1.bp.blogspot.com/-mDZD4P5yKyQ/WlPY-7f5kMI/AAAAAAAACZs/1K9HGOfUn0c1GR6Os5yXY5ygkd48YGgswCLcBGAs/s400/informationscreen.png" width="400" /></a></div>
</li>
<li id="line59">
<div id="line59">
Exit recovery.<br />
Press <code>OK</code> to exit recovery mode and continue booting.</div>
</li>
</ol>
<div class="blogger-post-footer"><script type="text/javascript"><!--
google_ad_client = "pub-1640070864477148";
//728x90, created 11/11/07
google_ad_slot = "6009857287";
google_ad_width = 728;
google_ad_height = 90;
//--></script>
<script type="text/javascript"
src="http://pagead2.googlesyndication.com/pagead/show_ads.js">
</script></div>Peter Leunghttp://www.blogger.com/profile/05589860210899238688noreply@blogger.com0tag:blogger.com,1999:blog-8032022811235182759.post-36100356462508995582018-02-01T04:30:00.000-08:002018-03-01T09:32:15.645-08:00How to mount USB drives on VirtualBox<a href="https://4.bp.blogspot.com/-bDT9SDcPTWM/Wjs4EPMA08I/AAAAAAAACSw/OCYRzXCpeFIrgpx2boPPtNkM2X7D9O2MgCLcBGAs/s1600/vbox.png" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="180" data-original-width="140" height="320" src="https://4.bp.blogspot.com/-bDT9SDcPTWM/Wjs4EPMA08I/AAAAAAAACSw/OCYRzXCpeFIrgpx2boPPtNkM2X7D9O2MgCLcBGAs/s320/vbox.png" width="249" /></a>
<br />
<div id="line0">
This is part 3 of the 4-part series on VirtualBox. This post shows you how to access, from your guest OS, a flash drive mounted on the Debian host. Parts <a href="https://linuxcommando.blogspot.com/2017/12/using-virtualbox-to-build-freebsd-vm-on.html">1,</a> <a href="https://www.blogger.com/blogger.g?blogID=8032022811235182759">2</a> and <a href="https://linuxcommando.blogspot.com/2018/03/sharing-folders-between-virtualbox-host.html">4</a> demonstrate how to build a FreeBSD virtual machine using VirtualBox on a Debian host, perform post-installation tasks, and share folders.</div><br />
<h2>
<a href="https://www.blogger.com/null" id="2"></a>Prerequisites</h2>
<div id="line4">
The VirtualBox extension pack must be installed on the VirtualBox host. See <a href="https://www.blogger.com/blogger.g?blogID=8032022811235182759">part 2</a> for instructions.</div><br />
<h2>
<a href="https://www.blogger.com/null" id="6"></a>Configuring USB</h2>
<div id="line8">
Follow the steps below to configure VirtualBox USB.</div>
<ol id="line10">
<li id="line10">
<div id="line10">
Add user to <code>vboxusers</code> group.<br />
VirtualBox access to the host's USB drives is only granted to users of the <code>vboxusers</code> group. As <code>root</code> on the host, run the following command to add each VirtualBox user (e.g., <code>peter</code>) to the group.</div>
<pre><code> # usermod -aG vboxusers peter
</code></pre>
</li>
<li id="line15">
<div id="line15">
Power off VM.<br />
VirtualBox defaults to using USB Controller 1.1 (OHCI). Modern hardware uses USB Controller 2.0(EHCI) and USB Controller 3.0(xHCI). Before you can change the USB controller protocol, the virtual machine must be powered off.</div>
</li>
<li id="line18">
<div id="line18">
Open <code>VirtualBox Manager</code>, click <code>Settings</code>, and select <code>USB</code>.</div>
</li>
<a href="https://1.bp.blogspot.com/-CVnxfyVJ6tY/Wk2q5D_xzuI/AAAAAAAACX0/O0FMANkJLgAYaxCGTjfLDBzKaEABb6augCLcBGAs/s1600/selectUSBCtrl.png" imageanchor="1"><img border="0" data-original-height="456" data-original-width="706" height="207" src="https://1.bp.blogspot.com/-CVnxfyVJ6tY/Wk2q5D_xzuI/AAAAAAAACX0/O0FMANkJLgAYaxCGTjfLDBzKaEABb6augCLcBGAs/s320/selectUSBCtrl.png" width="320" /></a>
<li id="line20">
<div id="line20">
Specify USB Controller.<br />
Select either <code>USB 2.0 (EHCI) Controller</code> or <code>USB 3.0 (xHCI) Controller</code> according to your actual hardware. <br />
<br />
Note that you can add <code>USB Device Filters</code> to define the types of USB drives which will be automatically made visible to the guest OS. Be forewarned that the USB drive, once made visible to the guest OS, will no longer be available to the host. More on the use of device filters in the next section.</div>
</li>
</ol><br />
<h2>
<a href="https://www.blogger.com/null" id="23"></a>Accessing USB drive</h2>
<div id="line25">
Below is the step-by-step procedure to mount and access a flash drive.</div>
<ol id="line27">
<li id="line27">
<div id="line27">
Insert the flash drive into your host machine's USB port.</div>
</li>
<li id="line28">
<div id="line28">
Unmount the flash drive (if it is auto-mounted on your host).<br />
Making it available to the guest will automatically and instantly unmount it from the host. To avoid any data loss due to pending writes to the drive, it is a good practice to explicitly unmount the drive prior to handing control to the guest.</div>
</li>
<li id="line30">
<div id="line30">
Power on the FreeBSD guest.</div>
</li>
<li id="line31">
<div id="line31">
Assign USB drive to guest OS.<br />
<a href="https://4.bp.blogspot.com/-DUzCy7uvtOY/Wk55YWNtp8I/AAAAAAAACYE/bQebkw76cs0EAXHFj-MLDQ9P9S9tmx5ywCLcBGAs/s1600/flashicon.png" imageanchor="1"><img border="0" data-original-height="554" data-original-width="646" height="274" src="https://4.bp.blogspot.com/-DUzCy7uvtOY/Wk55YWNtp8I/AAAAAAAACYE/bQebkw76cs0EAXHFj-MLDQ9P9S9tmx5ywCLcBGAs/s320/flashicon.png" width="320" /></a><br />
Open the virtual system console, and right click the <code>USB drive</code> icon.<br />
<a href="https://1.bp.blogspot.com/-_qLFUtTAw00/Wk6DO5CGNoI/AAAAAAAACYU/Y6xdaJXeP8c0ZA4ksaDgxm6p2_KtsuIawCLcBGAs/s1600/usboptions.png" imageanchor="1"><img border="0" data-original-height="555" data-original-width="790" height="225" src="https://1.bp.blogspot.com/-_qLFUtTAw00/Wk6DO5CGNoI/AAAAAAAACYU/Y6xdaJXeP8c0ZA4ksaDgxm6p2_KtsuIawCLcBGAs/s320/usboptions.png" width="320" /></a><br />
Click to select your USB drive.<br />
<br />
Note that this is a 1-time assignment only. Please see instructions at the end of the section on how to automatically assign this particular USB drive for all subsequent sessions.</div>
</li>
<li id="line35">
<div id="line35">
Login to FreeBSD, and mount the drive.<br />
You can mount a MS-DOS based flash drive by running the following commands as root. Replace <code>/dev/da0s1</code> with the proper device identifier for your USB drive. (You can find out the exact device ID by first running <code>dmesg</code> to identify the device name, e.g. <code>da0</code>, and <code>fdisk</code> to reveal the disk partition structure, e.g., <code>s1</code>.)</div>
<pre><code> # mkdir -p /media/usb
# mount -t msdosfs /dev/da0s1 /media/usb
</code></pre>
<div id="line41">
To unmount the drive,</div>
<pre><code> # umount /media/usb
</code></pre>
</li>
</ol>
<div id="line45">
To always automatically assign a particular USB drive to the guest OS, open the <code>VirtualBox Manager</code>, click <code>Settings</code>, and then <code>USB</code>.</div>
<a href="https://4.bp.blogspot.com/-6XVMdU4yTEg/Wk6HWUQmycI/AAAAAAAACYg/mC60_8TmneERe-y72Un4NF0O45oy1jjBACLcBGAs/s1600/alwaysattachusb.png" imageanchor="1"><img border="0" data-original-height="456" data-original-width="706" height="207" src="https://4.bp.blogspot.com/-6XVMdU4yTEg/Wk6HWUQmycI/AAAAAAAACYg/mC60_8TmneERe-y72Un4NF0O45oy1jjBACLcBGAs/s320/alwaysattachusb.png" width="320" /></a>
<br />
<div id="line47">
Finally, click <code>Add USB device filter (with the + sign)</code> icon, and select the USB drive that is currently inserted in the host.</div><br />
<h2>
<a href="https://www.blogger.com/null" id="49"></a>Related posts</h2>
<ul id="line50">
<li id="line50"><a href="https://linuxcommando.blogspot.com/2017/12/using-virtualbox-to-build-freebsd-vm-on.html">Part 1 of VirtualBox series - on installation</a>.</li>
<li id="line51"><a href="https://linuxcommando.blogspot.com/2018/01/what-to-do-after-installing-virtualbox.html">Part 2 - post-installation</a>.</li>
<li><a href="https://linuxcommando.blogspot.com/2018/03/sharing-folders-between-virtualbox-host.html">Part 4 - on sharing folders</a>.</li>
</ul>
<div class="blogger-post-footer"><script type="text/javascript"><!--
google_ad_client = "pub-1640070864477148";
//728x90, created 11/11/07
google_ad_slot = "6009857287";
google_ad_width = 728;
google_ad_height = 90;
//--></script>
<script type="text/javascript"
src="http://pagead2.googlesyndication.com/pagead/show_ads.js">
</script></div>Peter Leunghttp://www.blogger.com/profile/05589860210899238688noreply@blogger.com0tag:blogger.com,1999:blog-8032022811235182759.post-12379896594377357802018-01-16T10:30:00.000-08:002018-03-01T09:21:19.763-08:00What to do after installing VirtualBox<div id="line0">
<a href="https://4.bp.blogspot.com/-bDT9SDcPTWM/Wjs4EPMA08I/AAAAAAAACSw/OCYRzXCpeFIrgpx2boPPtNkM2X7D9O2MgCLcBGAs/s1600/vbox.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="180" data-original-width="140" height="320" src="https://4.bp.blogspot.com/-bDT9SDcPTWM/Wjs4EPMA08I/AAAAAAAACSw/OCYRzXCpeFIrgpx2boPPtNkM2X7D9O2MgCLcBGAs/s320/vbox.png" width="249" /></a>This post is part 2 of the 4-part series on creating a virtual machine using VirtualBox hosted on a Debian machine. <a href="https://linuxcommando.blogspot.com/2017/12/using-virtualbox-to-build-freebsd-vm-on.html">Part 1</a> focuses on the installation of VirtualBox and the guest OS (FreeBSD). Part 2 addresses things that you should do after installing VirtualBox. Lastly, <a href="https://linuxcommando.blogspot.com/2018/02/how-to-mount-usb-drives-on-virtualbox.html">parts 3</a> and <a href="https://linuxcommando.blogspot.com/2018/03/sharing-folders-between-virtualbox-host.html">4</a> deals specifically with accessing USB flash drives from the guest OS, and sharing folders.</div><br />
<h2>
<a href="https://www.blogger.com/null" id="2"></a>Install extension pack</h2>
<div id="line3">
The base VirtualBox package is missing some <a href="https://www.virtualbox.org/manual/ch01.html#intro-installing">important functionalities</a>: support for virtual USB 2.0 and 3.0 devices, host webcam passthrough, Intel PXE boot, and disk image encryption. To obtain those functionalities, you need to download and install the VirtualBox extension pack.</div>
<div id="line5">
Note that the extension pack operates under a much stricter license than the GPLv2 of the base VirtualBox package. Specifically, the VirtualBox Extension Pack Personal Use and Evaluation License (PUEL) 'is a free license for personal, educational or evaluation use'. For commercial use, you do have to pay a fee to Oracle.</div>
<div id="line7">
The procedure to install the extension pack is as follows:</div>
<ol id="line8">
<li id="line8">Download the VirtualBox extension pack to your host machine.<br />
<br />
The extension pack is available from the <a href="https://www.virtualbox.org/wiki/Downloads">VirtualBox website</a>. The package is universal for all host and guest OSes. However, you should use the same version for both the base and extension pack. In other words, when you upgrade your base, you should also upgrade the extension pack.</li>
<li id="line10">Run <code>VirtualBox Manager</code>.</li>
</ol>
<div id="line12">
<a href="https://2.bp.blogspot.com/-w537EtCKRZA/Wkw0Ge_L2KI/AAAAAAAACWs/QitwvDnIUK09Ja6WJO3rJSZvXMlJS6RogCLcBGAs/s1600/vbmanager.png" imageanchor="1"><img border="0" data-original-height="621" data-original-width="847" height="235" src="https://2.bp.blogspot.com/-w537EtCKRZA/Wkw0Ge_L2KI/AAAAAAAACWs/QitwvDnIUK09Ja6WJO3rJSZvXMlJS6RogCLcBGAs/s320/vbmanager.png" width="320" /></a></div>
<ol id="line14" start="3">
<li id="line14">Add extension pack.<br />
<br />
From the <code>File</code> menu, click <code>Preferences</code>. Then, select <code>Extensions</code> in the side panel.</li>
</ol>
<div id="line17">
<a href="https://4.bp.blogspot.com/-571He7F4A84/Wkw3-SdddnI/AAAAAAAACW4/2Bcu6ujS-yEe19JjtLrZNZ6rJSxHQLkeACLcBGAs/s1600/addextension.png" imageanchor="1"><img border="0" data-original-height="374" data-original-width="545" height="220" src="https://4.bp.blogspot.com/-571He7F4A84/Wkw3-SdddnI/AAAAAAAACW4/2Bcu6ujS-yEe19JjtLrZNZ6rJSxHQLkeACLcBGAs/s320/addextension.png" width="320" /></a></div>
<br />
Click the + icon to specify the location of the extension pack file (Oracle_VM_VirtualBox_Extension_Pack-5.2.4-119785.vbox-extpack). <br />
<br />
<div id="line21">
<a href="https://1.bp.blogspot.com/-VNP5wmALIiI/Wkw6Ud2ac-I/AAAAAAAACXE/NF6MlSkEQjganBWkQEZ9Bb6aRS8u6OjcwCLcBGAs/s1600/installpack.png" imageanchor="1"><img border="0" data-original-height="280" data-original-width="458" height="196" src="https://1.bp.blogspot.com/-VNP5wmALIiI/Wkw6Ud2ac-I/AAAAAAAACXE/NF6MlSkEQjganBWkQEZ9Bb6aRS8u6OjcwCLcBGAs/s320/installpack.png" width="320" /></a></div>
<div id="line23">
<br />
At the end, click <code>Install</code>. Before it begins installation, you will be prompted to accept the software license, and to enter the administrative password.</div>
<br />
<h2>
<a href="https://www.blogger.com/null" id="25"></a>Bridge host & guest networks</h2>
<div id="line26">
By default, the virtual machine is NATed (not bridged) to your host network. Consequently, the FreeBSD guest is on a different subnet than the host. For example, my FreeBSD guest has a <code>10.0.0.2</code> IP address while my Linux host has a <code>192.168.1.49</code> IP address. Practically, it means that I cannot ssh to the guest OS from anywhere in my host network. To make that possible, I need to bridge the host network and the guest. The procedure is below (the VM does <strong>not</strong> have to be powered off for you to change networking).</div>
<ol id="line28">
<li id="line28">Open the <code>VirtualBox Manager</code>, and click <code>Settings</code>.</li>
<li id="line29">Click <code>Network</code> in the side menu.</li>
</ol>
<div id="line31">
<a href="https://2.bp.blogspot.com/-rOO6KrjDRHE/Wk0v_ncU2iI/AAAAAAAACXk/9WF-Mn29WAw3zOidpoSxebdwtNVvXO-fwCLcBGAs/s1600/bridgevsnat.png" imageanchor="1"><img border="0" data-original-height="456" data-original-width="706" height="207" src="https://2.bp.blogspot.com/-rOO6KrjDRHE/Wk0v_ncU2iI/AAAAAAAACXk/9WF-Mn29WAw3zOidpoSxebdwtNVvXO-fwCLcBGAs/s320/bridgevsnat.png" width="320" /></a></div>
<ol id="line33" start="3">
<li id="line33">Select <code>Bridged Adapter</code> in the <code>Attached to</code> dropdown menu.</li>
</ol>
<h2>
<a href="https://www.blogger.com/null" id="35"></a>Enable copy & paste between host & guest</h2>
<div id="line36">
I administer the FreeBSD guest extensively from its virtual system console running on the Linux host. By default, the clipboard is not shared between the host and the guest systems. That means I often need to manually re-enter commands on the guest OS rather than copy and paste from the host. To share the system clipboard (and save yourself plenty of laborious typing), follow the steps below:</div>
<ol id="line37">
<li id="line37">Open the <code>VirtualBox Manager</code>, and click <code>Settings</code>.</li>
<li id="line38">Select <code>General</code>, and click the <code>Advanced</code> tab.</li>
</ol>
<div id="line40">
<a href="https://1.bp.blogspot.com/-zBBN6fxGaMA/Wk0nxkY2yTI/AAAAAAAACXU/akkyKiabbqo1ZHM_1rTR5yszJoMuS8WywCLcBGAs/s1600/copynpaste.png" imageanchor="1"><img border="0" data-original-height="456" data-original-width="706" height="207" src="https://1.bp.blogspot.com/-zBBN6fxGaMA/Wk0nxkY2yTI/AAAAAAAACXU/akkyKiabbqo1ZHM_1rTR5yszJoMuS8WywCLcBGAs/s320/copynpaste.png" width="320" /></a></div>
<ol id="line42" start="3">
<li id="line42">Select <code>bidirectional</code> for both <code>Shared Clipboard</code> and <code>Drag and Drop</code>.</li>
</ol>
<div id="line44">
Note that, to share the clipboard, the FreeBSD-specific guest additions package (<code>emulators/virtualbox-ose-additions</code>) must also be installed. The instruction for that is already detailed in the <code>Post-installation</code> section of <a href="https://linuxcommando.blogspot.com/2017/12/using-virtualbox-to-build-freebsd-vm-on.html">part 1</a> of this series.</div>
<br />
<h2>
<a href="https://www.blogger.com/null" id="46"></a>Related posts</h2>
<ul id="line47">
<li id="line47"><a href="https://linuxcommando.blogspot.com/2017/12/using-virtualbox-to-build-freebsd-vm-on.html">Part 1 of VirtualBox series</a>.</li>
<li id="line48"><a href="https://linuxcommando.blogspot.com/2018/02/how-to-mount-usb-drives-on-virtualbox.html">Part 3</a> - on mounting USB drives.</li>
<li><a href="https://linuxcommando.blogspot.com/2018/03/sharing-folders-between-virtualbox-host.html">Part 4</a> - on sharing folders.</li>
</ul>
<div class="blogger-post-footer"><script type="text/javascript"><!--
google_ad_client = "pub-1640070864477148";
//728x90, created 11/11/07
google_ad_slot = "6009857287";
google_ad_width = 728;
google_ad_height = 90;
//--></script>
<script type="text/javascript"
src="http://pagead2.googlesyndication.com/pagead/show_ads.js">
</script></div>Peter Leunghttp://www.blogger.com/profile/05589860210899238688noreply@blogger.com0tag:blogger.com,1999:blog-8032022811235182759.post-2346206678530788132018-01-02T12:30:00.000-08:002018-01-02T12:30:04.062-08:00axi-cache: a new search tool for Debian packages<br />
<div class="separator" style="clear: both; text-align: center;"><a href="https://4.bp.blogspot.com/-zYYQNBxww5Q/WkaDEzat2WI/AAAAAAAACV4/cICZKKO69_0hOIUswWDpvFTckcuVllUcQCLcBGAs/s1600/icons-842865_1280.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="1280" data-original-width="1280" height="200" src="https://4.bp.blogspot.com/-zYYQNBxww5Q/WkaDEzat2WI/AAAAAAAACV4/cICZKKO69_0hOIUswWDpvFTckcuVllUcQCLcBGAs/s200/icons-842865_1280.png" width="200" /></a></div>Debian has no shortage of tools when it comes to searching for packages. Revered oldtimers include <code>apt-cache</code>, <code>apt</code>, and <code>apt-file</code>. <code>axi-cache</code> is the new kid on the block. This post explains what is novel about <code>axi-cache</code> and how to use it. <br />
<br />
First, <code>axi-cache</code> needs to be installed and initialized as follows:<br />
<code><br />
# apt-get install apt-xapian-index<br />
# update-apt-xapian-index<br />
The index /var/lib/apt-xapian-index is up to date<br />
</code><br />
The basic <code>axi-cache</code> search is syntactically very similar with the aforementioned search commands:<br />
<code><br />
$ axi-cache search browser<br />
930 results found.<br />
Results 1-20:<br />
100% chromium-driver - web browser - WebDriver support<br />
99% chromium - web browser<br />
99% ruby-browser - browser detection for Ruby<br />
97% libwwwbrowser-perl - Platform independent means to start a WWW browser<br />
96% libhtml-display-perl - module for displaying HTML locally in a browser<br />
96% gcu-plugin - GNOME chemistry utils (browser plugin)<br />
96% python-zope.browser - Shared Zope Toolkit browser components<br />
96% python3-zope.browser - Shared Zope Toolkit browser components<br />
96% chromedriver - web browser - WebDriver support transitional package<br />
96% mythbrowser - Small web browser module for MythTV<br />
96% chromium-widevine - web browser - widevine content decryption support<br />
96% swfdec-mozilla - dummy package for transition to browser-plugin-gnash<br />
95% python-livereload - automatic browser refresher<br />
95% mozilla-plugin-gnash - dummy package for renaming to browser-plugin-gnash<br />
95% python3-livereload - automatic browser refresher (Python 3)<br />
95% qupzilla - lightweight web browser based on libqtwebkit<br />
95% python-livereload-doc - automatic browser refresher (documentation)<br />
95% firefox-esr - Mozilla Firefox web browser - Extended Support Release (ESR)<br />
95% ipig - integrating PSMs into genome browser visualisations<br />
94% epiphany-browser-data - Data files for the GNOME web browser<br />
More terms: refresher livereload safer stable refresh browsing webdriver<br />
More tags: field::religion culture::dutch uitoolkit::gtk field::chemistry use::browsing web::browser interface::x11<br />
`axi-cache more' will give more results<br />
</code><br />
A similar search using <code>apt-cache</code> returns 962 results with the top 20 results being: <br />
<code><br />
389-admin - 389 Directory Administration Server<br />
libds-admin-serv0 - Libraries for the 389 Directory Administration Server<br />
xul-ext-adblock-plus - advertisement blocking extension for web browsers<br />
ajaxterm - Web based terminal written in Python<br />
alevt - X11 Teletext/Videotext browser<br />
alice - Web browser (WebKit or Gecko) based IRC client<br />
xul-ext-all-in-one-sidebar - sidebar extension for Firefox<br />
node-almond - minimal AMD API implementation for use in optimized browser builds<br />
pilot - Simple file browser from Alpine, a text-based email client<br />
ams - Realtime modular synthesizer for ALSA<br />
amule-gnome-support - ed2k links handling support for GNOME web browsers<br />
libjs-angularjs - lets you write client-side web applications as if you had a smarter browser<br />
libjs-animate.css - cross-browser library of CSS animations<br />
libapache2-mod-upload-progress - upload progress support for the Apache web server<br />
apachedex - Compute APDEX from Apache-style logs<br />
xfonts-kapl - APL fonts for A+ development<br />
artemis - genome browser and annotation tool<br />
libjs-asciimathml - Library to render high quality mathematical formulas in a browser<br />
aspectj - aspect-oriented extension for Java - tools<br />
auctex - integrated document editing environment for TeX etc.<br />
</code><br />
<div class="separator" style="clear: both; text-align: center;"><a href="https://4.bp.blogspot.com/-jWF7OjzaBZw/WkaEduq5VHI/AAAAAAAACWE/IvVJDwNv4nczQP_pjRtVepLYTyxao0zhACLcBGAs/s1600/linux-24945_1280.png" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" data-original-height="1100" data-original-width="1280" height="172" src="https://4.bp.blogspot.com/-jWF7OjzaBZw/WkaEduq5VHI/AAAAAAAACWE/IvVJDwNv4nczQP_pjRtVepLYTyxao0zhACLcBGAs/s200/linux-24945_1280.png" width="200" /></a></div>Note that, unlike <code>apt-cache</code>, <code>axi-cache</code> returns, by default, only the top 20 hits. You can see the entire result set by specifying the <code>--all</code> option (e.g., <code>axi-cache --all search browser</code>). Alternatively, you can page through the results by running the following command after the initial search.<br />
<code><br />
$ axi-cache more<br />
</code> <br />
In general, <code>axi-cache</code> returns more relevant results than <code>apt-cache</code>. The latter implements a rudimentary <code>grep</code>-like search by matching regular-expression text patterns against the package name and description of a package. Unless you have some idea of the package's name, an <code>apt-cache</code> search often returns many irrelevant results, as indicated by the above example. In contrast, <code>axi-cache</code> can rank the search results by relevance with the help of the <a href="https://raphaelhertzog.com/2010/11/29/how-to-find-the-right-debian-packages-high-level-search-interface/"><i>A</i>pt <i>X</i>apian <i>I</i>ndex</a>(axi). This index is a database of package meta-data which includes much more than just a package's name and description. To examine what is indexed, run the following command:<br />
<code><br />
$ axi-cache info<br />
...<snipped>...<br />
Plugin status:<br />
aliases enabled, up to date (430 days, 15:47:18.356149 older than index)<br />
app-install disabled<br />
apttags enabled, needs indexing (6 days, 23:59:10.729973 newer than index)<br />
cataloged_time enabled, needs indexing (6 days, 23:59:10.729973 newer than index)<br />
debtags disabled<br />
descriptions enabled, needs indexing (6 days, 23:59:10.729973 newer than index)<br />
relations enabled, needs indexing (6 days, 23:59:10.729973 newer than index)<br />
sections enabled, needs indexing (6 days, 23:59:10.729973 newer than index)<br />
sizes enabled, needs indexing (6 days, 23:59:10.729973 newer than index)<br />
template enabled, up to date<br />
translated-desc enabled, needs indexing (1 day, 10:56:58.643851 newer than index)<br />
...<snipped>...<br />
</code><br />
As an aside, you can update the Apt Xapian Index by executing the following command as <code>root</code>:<br />
<code><br />
# update-apt-xapian-index <br />
</code><br />
The most distinctive feature of the index is its use of tags (<code>apttags</code>). These tags categorize a package by predefined facets such as <code>role</code>, <code>protocol</code>, <code>suite</code>, <code>culture</code>, <code>use</code>, <code>works-with</code>, etc.<br />
<br />
Suppose you are multi-lingual and you are looking for packages that install, say, Chinese fonts. You can quickly identify the packages you need using the following command.<br />
<code><br />
$ axi-cache search x11::font and culture::chinese and role::data<br />
13 results found.<br />
Results 1-13:<br />
100% fonts-arphic-bkai00mp - "AR PL KaitiM Big5" Chinese TrueType font by Arphic Technology<br />
100% fonts-arphic-bsmi00lp - "AR PL Mingti2L Big5" Chinese TrueType font by Arphic Technology<br />
100% fonts-arphic-gbsn00lp - "AR PL SungtiL GB" Chinese TrueType font by Arphic Technology<br />
100% fonts-arphic-gkai00mp - "AR PL KaitiM GB" Chinese TrueType font by Arphic Technology<br />
100% fonts-cwtex-fs - TrueType Font from cwTeX - FangSong<br />
100% fonts-cwtex-heib - TrueType Font from cwTeX - HeiBold<br />
100% fonts-cwtex-kai - TrueType Font from cwTeX - Kai<br />
100% fonts-cwtex-ming - TrueType Font from cwTeX - Ming<br />
100% fonts-cwtex-yen - TrueType Font from cwTeX - Yen<br />
100% ttf-wqy-zenhei - transitional dummy package<br />
100% xfonts-intl-chinese - international fonts for X - Chinese<br />
100% xfonts-intl-chinese-big - international fonts for X - large Chinese<br />
100% xfonts-unifont - PCF (bitmap) version of GNU Unifont<br />
More terms: chinese fonts truetype cwtex font koanughi cwttf<br />
More tags: made-of::font culture::taiwanese role::dummy role::app-data culture::greek culture::korean culture::russian<br />
</code><br />
Note that <code>axi-cache</code> supports logical operations such as <code>AND</code>, <code>OR</code>, and <code>NOT</code>.<br />
<br />
The following table compiles a non-exhaustive list of common facets and example values. For a complete list, please click <a href="https://anonscm.debian.org/cgit/debtags/vocabulary.git/tree/debian-packages">here</a>. To make a tag, join the facet and the value using 2 colons, e.g., <code>role::program</code>.<br />
<style type="text/css">
.tg {border-collapse:collapse;border-spacing:0;}
.tg td{font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;}
.tg th{font-family:Arial, sans-serif;font-size:14px;font-weight:normal;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;}
.tg .tg-yw4l{vertical-align:top}
</style><br />
<table class="tg"><tr> <th class="tg-yw4l">Facet</th> <th class="tg-yw4l">Values</th> </tr>
<tr> <td class="tg-yw4l">admin</td> <td class="tg-yw4l">filesystem, forensics, monitoring, power-management, virtualization</td> </tr>
<tr> <td class="tg-yw4l">culture</td> <td class="tg-yw4l">chinese, latvian, russian</td> </tr>
<tr> <td class="tg-yw4l">field</td> <td class="tg-yw4l">arts, astronomy, finance, mathematics, medicine, statistics</td> </tr>
<tr> <td class="tg-yw4l">game</td> <td class="tg-yw4l">arcade, board, card, fps, mud, puzzle, rpg, sport, toys, typing</td> </tr>
<tr> <td class="tg-yw4l">implemented-in</td> <td class="tg-yw4l">c, php</td> </tr>
<tr> <td class="tg-yw4l">interface</td> <td class="tg-yw4l">3d, commandline, graphical, shell, x11</td> </tr>
<tr> <td class="tg-yw4l">network</td> <td class="tg-yw4l">client, configuration, server, service, vpn</td> </tr>
<tr> <td class="tg-yw4l">office</td> <td class="tg-yw4l">finance, groupware, presentation, spreadsheet</td> </tr>
<tr> <td class="tg-yw4l">protocol</td> <td class="tg-yw4l">ip, ipv6, smtp, webdav</td> </tr>
<tr> <td class="tg-yw4l">role</td> <td class="tg-yw4l">app-data, data, debug-symbols, devel-lib, documentation, kernel, metapackage, plugin, program, shared-lib, source</td> </tr>
<tr> <td class="tg-yw4l">security</td> <td class="tg-yw4l">antivirus, authentication, cryptography, log-analyzer</td> </tr>
<tr> <td class="tg-yw4l">suite</td> <td class="tg-yw4l">bsd, debian, eclipse, emacs, gnome, gnu, kde, mozilla, mysql, openoffice, openstack, postgresql, xfce, xmms2</td> </tr>
<tr> <td class="tg-yw4l">system</td> <td class="tg-yw4l">cloud, embedded, laptop, mobile, server, virtual</td> </tr>
<tr> <td class="tg-yw4l">uitoolkit</td> <td class="tg-yw4l">gtk, motif, ncurses, qt, sdl, tk, xlib</td> </tr>
<tr> <td class="tg-yw4l">use</td> <td class="tg-yw4l">analysing, browsing, calculating, chatting, checking, compressing, configuring, converting, downloading, driver, editing, entertaining, filtering, gameplaying, learning, login, measuring, monitor, organizing, playing, printing, routing, scanning, searching, simulating, storing, synchronizing, transmission, typesetting, viewing</td> </tr>
<tr> <td class="tg-yw4l">works-with</td> <td class="tg-yw4l">archive, audio, calendar, db, file, font, image, logfile, mail, network-traffic, spreadsheet, text, unicode, video</td> </tr>
<tr> <td class="tg-yw4l">x11</td> <td class="tg-yw4l">font, screensaver, theme, window-manager</td> </tr>
</table><br />
In closing, <code>axi-cache</code> is a good search tool for Debian packages because it usually gives you more relevant results. One caveat is warranted, however. Not all Debian packages have tags defined, which is particularly true for packages that are downloaded from third-party non-standard repositories.<br />
<div class="blogger-post-footer"><script type="text/javascript"><!--
google_ad_client = "pub-1640070864477148";
//728x90, created 11/11/07
google_ad_slot = "6009857287";
google_ad_width = 728;
google_ad_height = 90;
//--></script>
<script type="text/javascript"
src="http://pagead2.googlesyndication.com/pagead/show_ads.js">
</script></div>Peter Leunghttp://www.blogger.com/profile/05589860210899238688noreply@blogger.com0tag:blogger.com,1999:blog-8032022811235182759.post-33257415706671218032017-12-21T18:02:00.000-08:002020-05-22T13:34:49.593-07:00Using VirtualBox to build FreeBSD VM on Debian<br />
<div class="separator" style="clear: both; text-align: center;"><a href="https://4.bp.blogspot.com/-bDT9SDcPTWM/Wjs4EPMA08I/AAAAAAAACSw/OCYRzXCpeFIrgpx2boPPtNkM2X7D9O2MgCLcBGAs/s1600/vbox.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="180" data-original-width="140" height="320" src="https://4.bp.blogspot.com/-bDT9SDcPTWM/Wjs4EPMA08I/AAAAAAAACSw/OCYRzXCpeFIrgpx2boPPtNkM2X7D9O2MgCLcBGAs/s320/vbox.png" width="249" /></a></div>Many virtualization software (aka hypervisors) can run on the host operating system of Linux. Notable open-source software include <a href="https://www.virtualbox.org/">VirtualBox</a>, <a href="https://www.linux-kvm.org/">KVM</a>, and <a href="https://www.xenproject.org/">Xen</a>. As part 1 of a 4-part series, this post outlines how to 1) install <a href="https://www.virtualbox.org/">VirtualBox</a> on Debian 10 (buster), and 2) build a FreeBSD virtual machine. <a href="https://linuxcommando.blogspot.com/2018/01/what-to-do-after-installing-virtualbox.html">Parts 2</a>, <a href="https://linuxcommando.blogspot.com/2018/02/how-to-mount-usb-drives-on-virtualbox.html">3</a>, and <a href="https://linuxcommando.blogspot.com/2018/03/sharing-folders-between-virtualbox-host.html">4</a> cover, respectively, additional post-installation tasks, how to access USB flash drives on the guest OS, and how to share folders between host and guest.<br />
<br />
<h2>About VirtualBox</h2><br />
In 2010, Oracle acquired Sun Microsystems and, as a result, took over the ownership of several open-source technologies including VirtualBox and mySQL. Subsequently, VirtualBox was rebranded as Oracle VM VirtualBox, but the software has remained free and open-source.<br />
<br />
VirtualBox, like KVM, is a type-2 hypervisor, meaning that it runs as a process on the host OS (Linux). In contrast, XEN is a type-1 hypervisor which runs directly on top of the bare-metal hardware. Type-1 hypervisors have an inherent advantage over type-2 regarding performance. Yet, type-2 hypervisors, such as VirtualBox, benefit from the continual advancement of their underlying OS.<br />
<br />
Hypervisors typically support the same multitude of guest operating systems including Linux, BSD, Solaris, Mac OS X, and Windows. But, hypervisors vary in what host OSes they support. For instance, KVM only runs on Linux. VirtualBox, on the other hand, runs on <a href="https://www.virtualbox.org/manual/ch01.html#hostossupport">multiple host platforms</a> such as Linux, Solaris, macOS, and Windows.<br />
<br />
<h2>Prepare for installation</h2><br />
Perform the following steps prior to installing VirtualBox: <br />
<br />
1. Enable virtualization technology in BIOS. <br />
<br />
2. Download the guest OS (FreeBSD) ISO. <br />
<br />
<h3>Enable virtualization in BIOS</h3><br />
In 2005, chip manufacturers introduced the instruction set extensions Intel VT and AMD-V to support hardware virtualization. Before that, all virtualization was done with software. VirtualBox supports both software and hardware virtualization. If hardware virtualization support is not available on a machine, VirtualBox will switch to software virtualization. Note that without hardware virtualization, VirtualBox can only create 32-bit virtual machines. In other words, a 64-bit virtual machine is only possible if hardware support is available. <br />
<br />
To determine if your CPU supports hardware virtualization, run the following command: <br />
<br />
<code><br />
$ egrep '^flags.*(vmx|svm)' /proc/cpuinfo<br />
</code><br />
<br />
The above command searches the CPU flags register for either 'vmx' (Intel VT) or 'svm' (AMD-V). If the result is non-nil, your machine does have hardware-assisted virtualization.<br />
<br />
Even if your machine supports hardware virtualization, it does not necessarily mean that it is currently enabled. To confirm that it is enabled, reboot the machine and enter BIOS. Once inside the BIOS, look for an option to turn on virtualization technology.<br />
<br />
<h3>Download FreeBSD ISO</h3><br />
VirtualBox is a full-virtualization hypervisor(as opposed to para-virtualization), meaning that the guest OS, FreeBSD in this case, can be used as is, without modification. Having said that,<br />
you are responsible for downloading the guest OS yourself.<br />
<br />
You can download the latest installer image (release 11.1) from the <a href="https://www.freebsd.org/where.html">FreeBSD website</a>. You have the options of downloading regular Installer Images, Virtual Machine Images, and SD Card Images. For this tutorial, use regular Installer Images. (VirtualBox does not support all Virtual Machine image formats.) Make sure that you select the right hardware architecture (<a href="https://download.freebsd.org/ftp/releases/amd64/amd64/ISO-IMAGES/11.1/">amd64</a> vs <a href="https://download.freebsd.org/ftp/releases/i386/i386/ISO-IMAGES/11.1/">i386</a>).<br />
<br />
Within the architecture, download the compressed disk image file with the disc1.iso.xz suffix and the corresponding SHA512 checksum file. The .xz file is about 40% smaller in size than the uncompressed version and will save you download time.<br />
<br />
After you have downloaded the 2 files, decompress the disk image file and do a checksum using the respective commands below. <br />
<br />
<code><br />
$ unxz FreeBSD-11.1-RELEASE-amd64-disc1.iso.xz<br />
$ sha512sum -c CHECKSUM.SHA512-FreeBSD-11.1-RELEASE-amd64<br />
...<br />
FreeBSD-11.1-RELEASE-amd64-disc1.iso: OK<br />
...<br />
</code><br />
<br />
<h2>Install VirtualBox</h2><br />
The following commands need to be executed with root privilege.<br />
<br />
<ol><li>Add the source repository for VirtualBox. <br />
<br />
<code><br />
# echo deb http://download.virtualbox.org/virtualbox/debian buster contrib > /etc/apt/sources.list.d/virtualbox.list<br />
</code></li>
<li>Import the Oracle GPG public key. <br />
<br />
<code><br />
# wget -q https://www.virtualbox.org/download/oracle<em>vbox</em>2016.asc -O- | apt-key add -<br />
OK<br />
</code></li>
<li>Install VirtualBox. <br />
<br />
<code><br />
# apt update<br />
# apt install virtualbox-6.1<br />
</code></li>
</ol><br />
<h2>Create FreeBSD VM</h2><br />
After installing VirtualBox, run it for the first time by clicking <code>Oracle VM VirtualBox</code> in your menu system. For the Xfce desktop, the option is under the <code>System</code> submenu. Then, click <code>New</code> to create a virtual machine.<br />
<br />
<a href="https://2.bp.blogspot.com/-XGaelUNmcHA/WjwRScmJgtI/AAAAAAAACTk/p51k_jb5pNoGHYcOHHFUQmyyh4ZLHs8DwCLcBGAs/s1600/VirtualBoxEmpty.png" imageanchor="1"><img border="0" data-original-height="572" data-original-width="966" height="189" src="https://2.bp.blogspot.com/-XGaelUNmcHA/WjwRScmJgtI/AAAAAAAACTk/p51k_jb5pNoGHYcOHHFUQmyyh4ZLHs8DwCLcBGAs/s320/VirtualBoxEmpty.png" width="320" /></a><br />
<br />
Next, provide a host name, and specify the <code>Type</code> (<code>BSD</code>), and <code>Version</code> (<code>FreeBSD 64-bit</code> or <code>32-bit</code>).<br />
<br />
<a href="https://2.bp.blogspot.com/-lcwd7iyPUKE/WjwRc2JXwWI/AAAAAAAACTo/6nbyiel6oucVi7x-7g6caP6Av7xTv0CQACLcBGAs/s1600/VirtualBoxOS.png" imageanchor="1"><img border="0" data-original-height="403" data-original-width="551" height="234" src="https://2.bp.blogspot.com/-lcwd7iyPUKE/WjwRc2JXwWI/AAAAAAAACTo/6nbyiel6oucVi7x-7g6caP6Av7xTv0CQACLcBGAs/s320/VirtualBoxOS.png" width="320" /></a><br />
<br />
The remaining steps configure the memory size, create a virtual hard disk and specify its file format, its pathname and size, and whether storage is dynamic or fixed-sized. At each step, reasonable defaults are given, and can be accepted unless your requirements dictate otherwise. Click <code>Create</code> at the end to create the virtual machine.<br />
<br />
The virtual machine is in the powered-off state after creation. Next, power up the machine by clicking <code>Start</code>.<br />
<br />
<a href="https://4.bp.blogspot.com/-CwS8ZviNKWU/WjwzDEMWOPI/AAAAAAAACT4/w_5ujIEePHAkPR8kd96Ll6Nc7k8IROARACLcBGAs/s1600/VirtualBoxInstallOS.png" imageanchor="1"><img border="0" data-original-height="572" data-original-width="966" height="189" src="https://4.bp.blogspot.com/-CwS8ZviNKWU/WjwzDEMWOPI/AAAAAAAACT4/w_5ujIEePHAkPR8kd96Ll6Nc7k8IROARACLcBGAs/s320/VirtualBoxInstallOS.png" width="320" /></a><br />
<br />
At this point, the VM has no operating system. Powering it on prompts you to insert a start-up disk. Click the folder icon to specify the file path for the disk image file downloaded earlier (<code>FreeBSD-11.1-RELEASE-amd64-disc1.iso</code>).<br />
<br />
<a href="https://3.bp.blogspot.com/-BbXRwpfauko/Wjw5KF1TSNI/AAAAAAAACUI/10rhlSr00ggY3sIj9zn9aWEvAhIJMIy9QCLcBGAs/s1600/VirtualBoxStartUpDisk.png" imageanchor="1"><img border="0" data-original-height="391" data-original-width="534" height="234" src="https://3.bp.blogspot.com/-BbXRwpfauko/Wjw5KF1TSNI/AAAAAAAACUI/10rhlSr00ggY3sIj9zn9aWEvAhIJMIy9QCLcBGAs/s320/VirtualBoxStartUpDisk.png" width="320" /></a><br />
<br />
Continue by selecting <code>Start</code>. The FreeBSD boot-up screen appears. Hit <code>Enter</code> to boot.<br />
<br />
<a href="https://3.bp.blogspot.com/-ME74Njbx03A/Wjw-ju5QE8I/AAAAAAAACUY/m6FY-PQ2C6w7k6Tbhjj6d2K6j0UxEBz6wCLcBGAs/s1600/VirtualBoxBSDBoot.png" imageanchor="1"><img border="0" data-original-height="474" data-original-width="726" height="209" src="https://3.bp.blogspot.com/-ME74Njbx03A/Wjw-ju5QE8I/AAAAAAAACUY/m6FY-PQ2C6w7k6Tbhjj6d2K6j0UxEBz6wCLcBGAs/s320/VirtualBoxBSDBoot.png" width="320" /></a><br />
<br />
Select <code>Install</code> to continue installation.<br />
<br />
<a href="https://1.bp.blogspot.com/-9mQ1rebzfQ4/WjxGBvMLIlI/AAAAAAAACUo/GFiGTBtnsXE4MCGCXbYSKAKPqkQ1ahDJgCLcBGAs/s1600/VirtualBoxInstall.png" imageanchor="1"><img border="0" data-original-height="474" data-original-width="726" height="209" src="https://1.bp.blogspot.com/-9mQ1rebzfQ4/WjxGBvMLIlI/AAAAAAAACUo/GFiGTBtnsXE4MCGCXbYSKAKPqkQ1ahDJgCLcBGAs/s320/VirtualBoxInstall.png" width="320" /></a><br />
<br />
The FreeBSD installer now takes over, and prompts you for information it needs to configure the system.<br />
<br />
At the end, the installer will prompt you to reboot into the installed system. At this point, you should reboot, but first you must unmount the disk image file (otherwise, you will boot into the CD image again). The reboot procedure I recommend is as follows. First, close the VM window by clicking the X in the upper right-hand corner.<br />
<br />
<a href="https://3.bp.blogspot.com/-DghUU7k5L1w/WjxWnyUs5qI/AAAAAAAACVI/genBTGI4OZMGGZ7zH2E-NU_npoKfrFAzQCLcBGAs/s1600/VirtualBoxRebootSys.png" imageanchor="1"><img border="0" data-original-height="474" data-original-width="726" height="209" src="https://3.bp.blogspot.com/-DghUU7k5L1w/WjxWnyUs5qI/AAAAAAAACVI/genBTGI4OZMGGZ7zH2E-NU_npoKfrFAzQCLcBGAs/s320/VirtualBoxRebootSys.png" width="320" /></a><br />
<br />
In the ensuing dialog box, select <code>Power off the machine</code>.<br />
<br />
<a href="https://2.bp.blogspot.com/-K8oleR6jVYs/WjxXkS-N9VI/AAAAAAAACVQ/QHqZIjOOxCY7QpdzTtXWVq_wS48b_2T_wCLcBGAs/s1600/VirtualBoxPowerOff.png" imageanchor="1"><img border="0" data-original-height="179" data-original-width="283" height="202" src="https://2.bp.blogspot.com/-K8oleR6jVYs/WjxXkS-N9VI/AAAAAAAACVQ/QHqZIjOOxCY7QpdzTtXWVq_wS48b_2T_wCLcBGAs/s320/VirtualBoxPowerOff.png" width="320" /></a><br />
<br />
To unmount the CD disk image, click the <code>Machine Tools</code> down-arrow, and select <code>Details</code>.<br />
<br />
<a href="https://4.bp.blogspot.com/-LFfysg76JnM/WjxZEMSGiGI/AAAAAAAACVc/_E2CfpVhvw4yJ_BPq4ZRFGHVvjNgW2IKQCLcBGAs/s1600/VirtualBoxMachineTools.png" imageanchor="1"><img border="0" data-original-height="572" data-original-width="966" height="189" src="https://4.bp.blogspot.com/-LFfysg76JnM/WjxZEMSGiGI/AAAAAAAACVc/_E2CfpVhvw4yJ_BPq4ZRFGHVvjNgW2IKQCLcBGAs/s320/VirtualBoxMachineTools.png" width="320" /></a><br />
<br />
In the Storage section, right click the <code>Optical Drive</code> besides the <code>IDE Secondary Master</code> label, and select <code>Remove disk from optical drive</code>.<br />
<br />
<a href="https://4.bp.blogspot.com/-yPLMgBt__wQ/WjxbAxPpS-I/AAAAAAAACVo/S4fdAbLcC7UHaj0f9AgBv9ZSNb2DmexwwCLcBGAs/s1600/VirtualBoxUnmount.png" imageanchor="1"><img border="0" data-original-height="572" data-original-width="966" height="189" src="https://4.bp.blogspot.com/-yPLMgBt__wQ/WjxbAxPpS-I/AAAAAAAACVo/S4fdAbLcC7UHaj0f9AgBv9ZSNb2DmexwwCLcBGAs/s320/VirtualBoxUnmount.png" width="320" /></a><br />
<br />
Now that the CD image is unmounted, you can click <code>Start</code> in the upper left-hand corner to boot the VM.<br />
<br />
<h2>Post-installation</h2><br />
VirtualBox offers additional functionalities via guest-specific addition packages. Guest additions packages enable the sharing of folders and the clipboard between host and guest. Follow the procedure below to install VirtualBox guest additions for FreeBSD.<br />
<br />
<ol><li>Login to FreeBSD as <code>root</code>.</li>
<li>Install the VirtualBox guest addition packages. <br />
<br />
<code><br />
# pkg install emulators/virtualbox-ose-additions<br />
</code></li>
<li>Append the following 2 lines to the startup configuration file <code>/etc/rc.conf</code>. <br />
<br />
<code><br />
vboxguest<em>enable="YES" <br />
<br />
vboxservice</em>enable="YES" <br />
<br />
</code></li>
<li>Add each X11 user (say <code>peter</code>) to the <code>wheel</code> group. <br />
<br />
<code><br />
# pw groupmod wheel -m peter <br />
<br />
</code></li>
<li>Reboot the VM to have the changes take effect.</li>
</ol><br />
<h2>Summary & conclusion</h2><br />
VirtualBox is easy to install. With VirtualBox, creating a FreeBSD virtual machine on a Linux host machine is straightforward. If you are looking for a open-source multi-platform hypervisor, VirtualBox should be high on your priority list.<br />
<h2>Related posts</h2><ul><li><a href="https://linuxcommando.blogspot.com/2018/01/what-to-do-after-installing-virtualbox.html">Part 2 - post-installation</a>.</li>
<li><a href="https://linuxcommando.blogspot.com/2018/02/how-to-mount-usb-drives-on-virtualbox.html">Part 3</a> - on mounting USB drives.</li>
<li><a href="https://linuxcommando.blogspot.com/2018/03/sharing-folders-between-virtualbox-host.html">Part 4</a> - on sharing folders.</li>
</ul><div class="blogger-post-footer"><script type="text/javascript"><!--
google_ad_client = "pub-1640070864477148";
//728x90, created 11/11/07
google_ad_slot = "6009857287";
google_ad_width = 728;
google_ad_height = 90;
//--></script>
<script type="text/javascript"
src="http://pagead2.googlesyndication.com/pagead/show_ads.js">
</script></div>Peter Leunghttp://www.blogger.com/profile/05589860210899238688noreply@blogger.com1tag:blogger.com,1999:blog-8032022811235182759.post-39850178521446617572016-07-27T07:00:00.000-07:002020-03-20T10:50:56.953-07:00ExpressVPN protects your on-line anonymity and privacy<p>This post reviews <a href="https://www.expressvpn.com/">ExpressVPN</a>, a hosted Virtual Private Network (VPN) service. A hosted VPN service is a paid subscription service. With a VPN, all your Internet communication is encrypted and passed through a secure proxy (the VPN server) before continuing to the intended destination. To the rest of the world, the Internet traffic appears to come from the VPN server, not your home computer.</p><br />
<h2>Why VPN?</h2><p>Subscription to a VPN service provides many benefits. For a brief video introduction, please click <a href="https://www.youtube.com/watch?v=R8We--FWw7o">here</a>. </p><br />
<h3>Anonymity and privacy</h3><br />
<div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-hZuicAPq0BM/V5eRJd5lRxI/AAAAAAAACNc/MF7ADz-cWo4B6wopyG7Rnx7aE8y9_QVIQCLcB/s1600/keyboard-895556_640.jpg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://1.bp.blogspot.com/-hZuicAPq0BM/V5eRJd5lRxI/AAAAAAAACNc/MF7ADz-cWo4B6wopyG7Rnx7aE8y9_QVIQCLcB/s200/keyboard-895556_640.jpg" width="200" height="113" /></a></div><br />
<p>When you connect to the Internet, you are exposing yourself to the world of hackers and government spy agencies who want to track your on-line activities, and steal your private information. </p><p>Many people have misplaced their trust in the Internet Service Providers (ISP) to protect their on-line anonymity and privacy. It is generally well known that ISPs do log your Internet activities. They are obliged to hand over the data if they are subpoenaed by government authorities.</p><p>The tech savvy may run <a href="https://www.torproject.org/">Tor</a>, a popular security tool, on their home computers. But, running Tor by itself is not good enough. While the data is well protected, your ISP does know that you are using Tor. The mere usage of Tor may arouse suspicion and attract extra unwanted attention from the authorities.</p><p>By using VPN, all your Internet data and activities (including Tor usages) are protected, even from your ISP. The key is that the VPN vendor must not track your VPN traffic. This "no data logging" policy is written into ExpressVPN's <a href="https://www.expressvpn.com/tos">terms of service agreement</a>.</p><br />
<h3>Breaking censorship</h3><br />
<div class="separator" style="clear: both; text-align: center;"><a href="https://4.bp.blogspot.com/-ZPkhBJ2ZOZ0/V5ePudeEe2I/AAAAAAAACNU/Go__3psE-ZkHu5MpQ6-jLunBQeyRBmIbgCLcB/s1600/shield-107860_640.jpg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://4.bp.blogspot.com/-ZPkhBJ2ZOZ0/V5ePudeEe2I/AAAAAAAACNU/Go__3psE-ZkHu5MpQ6-jLunBQeyRBmIbgCLcB/s200/shield-107860_640.jpg" width="200" height="141" /></a></div><p>Even if you live in a democratic country, you may be subject to regional Internet restrictions. For example, I cannot watch NBC on-line because I live in Canada and they restrict access to American viewers only. Likewise, American viewers cannot stream hockey games from the Canadian CBC website.</p><p>With a VPN, you can break censorship by opening a VPN connection to a server located in the target country. For example, to watch the American NBC, I connect my computer to a VPN server located in the USA. In this way, my request to watch NBC is granted because it appears to come from an American location.</p><p>For better security and service to you, VPN vendors must deploy servers in as many cities and countries as possible. ExpressVPN's servers are <a href="https://www.expressvpn.com/vpn-server#locations">located</a> in 136 cities across 87 countries. This should cover the location requirement for most people.</p><br />
<h2>VPN service features</h2><br />
<h3>Pricing & payment options</h3><div class="separator" style="clear: both; text-align: center;"><a href="https://4.bp.blogspot.com/-NuZqEnp5t8k/V5eT2XFuWFI/AAAAAAAACNo/TtAUyuJBVAwBXgpwhM4mR0GC8Htw7VhhwCLcB/s1600/BC_nBG_64px.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://4.bp.blogspot.com/-NuZqEnp5t8k/V5eT2XFuWFI/AAAAAAAACNo/TtAUyuJBVAwBXgpwhM4mR0GC8Htw7VhhwCLcB/s200/BC_nBG_64px.png" width="200" height="76" /></a></div><p><a href="https://www.expressvpn.com/order">ExpressVPN's pricing</a> is not the cheapest in the industry. However, it does offer a flexible term: you can sign up for 1 month, or you save money by committing for 6 or 12 months. All plans come with a generous 30-day money-back guarantee, and 7x24 live chat support.</p><p>I like the <a href="https://www.expressvpn.com/order">payment options</a> that ExpressVPN offers. In addition to the major credit cards, ExpressVPN also accepts PayPal and Bitcoin. If you pay with a credit card, your identity is associated with your VPN account. On the other hand, you can buy bitcoins anonymously. If you pay with bitcoins which you purchased anonymously, you remain anonymous even to the VPN vendor.</p><h3>Linux support</h3><div class="separator" style="clear: both; text-align: center;"><a href="https://3.bp.blogspot.com/-QewEUjWimPY/V5ebKHbJlsI/AAAAAAAACN4/mUgIYwUhsnY3KeEiDesYeCTn74lF4mDYgCLcB/s1600/tux-154942_640.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://3.bp.blogspot.com/-QewEUjWimPY/V5ebKHbJlsI/AAAAAAAACN4/mUgIYwUhsnY3KeEiDesYeCTn74lF4mDYgCLcB/s200/tux-154942_640.png" width="200" height="180" /></a></div><p>Many VPN vendors claim support for Linux. Windows users can download a VPN client which automatically configures your VPN connection. In contrast, the degree of Linux support is often in the form of instructions on how to manually set up a VPN connection.</p><p>ExpressVPN's <a href="https://www.expressvpn.com/vpn-software/vpn-linux">Linux support</a> is exceptional. You can download the VPN client for major Linux distributions such as Debian, Ubuntu, Fedora, and CentOS. You use the client for all your day-to-day VPN operations, such as connecting and disconnecting from the VPN, listing available servers, and reporting the connection status.</p><p><br />
<br />
</p><h2>Installing ExpressVPN client</h2><br />
<p>After you sign up for the service, download the VPN client according to the instructions in the official welcome email. You need to specify the Linux distribution before the download can proceed. For Debian or Ubuntu, select Ubuntu (there is no Debian option per se). Similarly, for Fedora or Centos, select Fedora.</p><p>It is also a good idea to download the VPN client's<br />
signature file. For instructions on how to use the signature file to verify the client download, click <a href="https://www.expressvpn.com/support/vpn-setup/pgp-for-linux/">here</a>.</p><p>After you successfully download the client (<code>expressvpn_1.1.0_amd64.deb</code>), installing it is as easy as running the following command:</p><p><code><br />
$ sudo dpkg -i expressvpn_1.1.0_amd64.deb <br />
</code></p><p>Next, you need to activate the VPN service. Note that you only need to do it once. You will be prompted to enter the activation code as provided to you in the welcome email.</p><p><code><br />
$ expressvpn activate<br />
</code></p><p>You can download and install the VPN client on as many devices as you wish. But, you can only <br />
have a maximum of 3 simultaneous VPN connections.</p><p>The next section explains how you will use the ExpressVPN client.</p><h2>Useful commands</h2><p>To connect to the VPN, run this command:</p><p><code><br />
$ expressvpn connect <br />
Connecting to Smart Location... <br />
Connecting to Canada - Montreal - 2... 100.0% <br />
Connected. <br />
</code></p><p>Note that I did not specify which VPN server to connect to. When you connect for the very first time and you do not specify the server, it will default to a recommended server, the 'smart' server, e.g., Montreal. In subsequent connections, it will default to the previous server.</p><p>What if I don't want to connect to the Montreal server? I live in Vancouver which is about 4,000 kilometres (or 2,485 miles) away from Montreal. So, I want to connect to a nearer server.</p><p>To switch servers, follow the steps below.</p><ol><li> List the available servers using the following command. </li>
<pre>$ expressvpn list
ALIAS COUNTRY LOCATION RECOMMENDED
----- --------------- ----------------- -----------
smart Smart Location Canada-Montreal-2 Y
...
usny United States (US) USA - New York Y
...
usse USA - Seattle Y
...
</pre>Of all the servers, it turns out that Seattle is closest to Vancouver. Later, I will use the alias "usse" from column 1 as a short form for the Seattle server.
<li> Disconnect from the current server. </li>
<pre>$ expressvpn disconnect
</pre><li>Connect to the target server. </li>
<pre>$ expressvpn connect usse
Connecting to USA - Seattle... 100.0%
Connected.
</pre></ol><p>To verify the status of the VPN connection, run this command:</p><p><code><br />
$ expressvpn status <br />
Connected to USA - Seattle <br />
</code></p><br />
<h2>Performance test</h2><br />
<div class="separator" style="clear: both; text-align: center;"><a href="https://2.bp.blogspot.com/-c1El1ppwJ3E/V5fKajfEfdI/AAAAAAAACOI/D00HYRapjO8-Oo5wmkxl_mOKUIfd07SIgCLcB/s1600/speedometer-34134_640.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://2.bp.blogspot.com/-c1El1ppwJ3E/V5fKajfEfdI/AAAAAAAACOI/D00HYRapjO8-Oo5wmkxl_mOKUIfd07SIgCLcB/s200/speedometer-34134_640.png" width="200" height="200" /></a></div><br />
<p>A VPN service encrypts and reroutes your Internet traffic through the VPN server. Because of this indirection, it adds some level of overhead to the VPN speed performance.</p><p>To measure the performance overhead of ExpressVPN, I run the following tests.</p><ol><li><p>Baseline (No VPN)</p><p>I run 3 tests without VPN. A good test tool to use is <a href="https://www.speedcheck.org/">Speedcheck.org</a>. Each test measures the download and upload speeds. Results from the 3 tests are averaged and recorded in the row labeled 'No VPN' in the table below. </p></li>
<li><p>VPN connection to the nearest server (Seattle)</p><p>3 more tests were run with VPN connection to the Seattle server. Note that the download and upload speeds take a 21% and 18% hit respectively when you compare the results with tests performed without VPN. A drop in speed is unavoidable because of the inherent VPN performance overhead. This level of performance degradation is often acceptable to most users, and can be viewed as the cost of protecting your on-line privacy and anonymity.</p></li>
<li><p>VPN connection to the smart server (Montreal)</p><p>The Montreal server is located 4,000 kilometres (or 2,485 miles) away from Vancouver. In contrast, the Seattle server is only 200 kilometres (or 124 miles) away. In light of the greater distance, it is not surprising that the Montreal speed tests took a bigger hit than the Seattle tests.</p></li>
</ol><p><style type="text/css">
.tg {border-collapse:collapse;border-spacing:0;}
.tg td{font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;}
.tg th{font-family:Arial, sans-serif;font-size:14px;font-weight:normal;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;}
.tg .tg-yw4l{vertical-align:top}
</style></p><table class="tg"><tr> <th class="tg-yw4l">VPN status</th> <th class="tg-yw4l">Ave download speed (Mbps)</th> <th class="tg-yw4l">Download speed penalty (%) </th> <th class="tg-yw4l">Ave upload speed (Mbps)</th> <th class="tg-yw4l">Upload speed penalty (%) </th> </tr>
<tr> <td class="tg-yw4l">No VPN</td> <td class="tg-yw4l">26.73</td> <td class="tg-yw4l"> N/A</td> <td class="tg-yw4l">6.69</td> <td class="tg-yw4l">N/A</td> </tr>
<tr> <td class="tg-yw4l">Connected to USA - Seattle</td> <td class="tg-yw4l">21.10</td> <td class="tg-yw4l">21</td> <td class="tg-yw4l">5.49</td> <td class="tg-yw4l">18</td> </tr>
<tr> <td class="tg-yw4l">Connected to Canada - Montreal</td> <td class="tg-yw4l">18.80</td> <td class="tg-yw4l">30</td> <td class="tg-yw4l">5.40</td> <td class="tg-yw4l">19</td> </tr>
</table><br />
<h2>Summary & conclusion</h2><br />
<div class="separator" style="clear: both; text-align: center;"><a href="https://3.bp.blogspot.com/-nZgWrCASvjE/V4k-QMq1lNI/AAAAAAAACMY/CPITTdalAvofWh1nMVoTHr_ZAtblcIl-ACLcB/s1600/thumb-up-1426815_640.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://3.bp.blogspot.com/-nZgWrCASvjE/V4k-QMq1lNI/AAAAAAAACMY/CPITTdalAvofWh1nMVoTHr_ZAtblcIl-ACLcB/s200/thumb-up-1426815_640.png" width="193" height="200" /></a></div><br />
<h3>Pros</h3><br />
<ul><li><p>Excellent Linux command-line interface</p></li>
<li><p>30-day money back guarantee</p></li>
<li><p>7x24 customer support via live chat or email</p></li>
<li><p>PayPal, Bitcoin and many more payment method options</p></li>
</ul><br />
<div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-36aw0o3kRBQ/V4k-n8E5w9I/AAAAAAAACMc/0KCwmZOaqTUOb44mSFU7FonkyMuLpb3rgCLcB/s1600/thumb-down-1426814_640.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://1.bp.blogspot.com/-36aw0o3kRBQ/V4k-n8E5w9I/AAAAAAAACMc/0KCwmZOaqTUOb44mSFU7FonkyMuLpb3rgCLcB/s200/thumb-down-1426814_640.png" width="193" height="200" /></a></div><br />
<h3>Cons</h3><br />
<ul><li>Restricted number of devices for simultaneous connections</li>
</ul><br />
<p><br />
<br />
<br />
<br />
<br />
</p><p>There are many VPN solutions in the market. But, if you are looking for Linux support, you should definitely give ExpressVPN a try. Linux power users will enjoy the use of the command-line VPN client.</p><br />
<h3>Disclaimer</h3><p>Linuxcommando was provided a free ExpressVPN subscription for this review.</p><div class="blogger-post-footer"><script type="text/javascript"><!--
google_ad_client = "pub-1640070864477148";
//728x90, created 11/11/07
google_ad_slot = "6009857287";
google_ad_width = 728;
google_ad_height = 90;
//--></script>
<script type="text/javascript"
src="http://pagead2.googlesyndication.com/pagead/show_ads.js">
</script></div>Peter Leunghttp://www.blogger.com/profile/05589860210899238688noreply@blogger.com6tag:blogger.com,1999:blog-8032022811235182759.post-84203593781673347262016-01-26T11:49:00.000-08:002018-02-18T20:21:40.069-08:00Running bash commands in parallel<div class="separator" style="clear: both; text-align: center;"><a href="https://2.bp.blogspot.com/-zjDPFWPxnXk/VpPqw147-kI/AAAAAAAACKQ/EW6MZP7otYE/s1600/processor-583695_640.jpg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://2.bp.blogspot.com/-zjDPFWPxnXk/VpPqw147-kI/AAAAAAAACKQ/EW6MZP7otYE/s200/processor-583695_640.jpg" /></a></div><h2>Introduction</h2><br />
<p>A modern server is typically multi-core, perhaps even multi-CPU. That is plenty of computing power to unleash on a given job. However, unless you run a job in parallel, you are not maximizing the use of all that power.</p><br />
<p>Below are some typical everyday operations we can speed up using parallel computing:</p><ol><li> Backup files from multiple source directories to a removable disk.<br />
<li> Resize image files in a directory.<br />
<li> Compress files in a directory.<br />
</ol>
<p>To execute a job in parallel, you can use any of the following commands:</p>
<ul><li><code>ppss</code></li>
<li><code>pexec</code></li>
<li><code>GNU parallel</code></li>
</ul>
<p>This post focuses on the <a href="https://www.gnu.org/software/parallel/"><code>GNU parallel</code></a> command.</p>
<h2>Installation of <code>GNU parallel</code></h2>
<p>To install <code>GNU parallel</code> on a Debian/Ubuntu system, run the following command:</p>
<p><code>
$ sudo apt-get install parallel
</code></p>
<h2>General Usage</h2>
<p>The <code>GNU parallel</code> program provides many options which you can specify to customize its behavior.
Interested readers can read its <a href="https://www.gnu.org/software/parallel/man.html"><code>man</code> page</a> to learn more about their usage. In this post, I will narrow the execution of <code>GNU parallel</code> to the following scenario.</p>
<p>My objective is to run a shell command in parallel, but on the same multi-core machine. The command can take multiple options, but only 1 is variable. Specifically, you run concurrent instances of the command by providing a different value for that one variable option. The different values are fed, one per line, to <code>GNU parallel</code> via the standard input.</p>
<p>The rest of this post shows how <code>GNU parallel</code> can backup multiple source directories by running <code>rsync</code> in parallel.</p>
<h2>Parallel backup</h2>
<p>The following command backs up 2 directories in parallel: <code>/home/peter</code> and <code>/data</code>. </p>
<p><code>
$ echo -e '/home/peter\n/data' | parallel -j-2 -k --eta rsync -R -av {} /media/myBKUP
</code></p>
<h3>Standard input</h3>
<p>The <code>echo</code> command assembles the 2 source directory locations, separated by a newline character (<code>\n</code>), and pipes it to <code>GNU parallel</code>.</p>
<h3>How many jobs?</h3>
<p>By default, <code>GNU parallel</code> deploys 1 job per core. You can override the default usint the <code>-j</code> option.</p>
<p><code>-j</code> specifies the maximum number of parallel jobs that <code>GNU parallel</code> can deploy. The maximum number can be specified in 1 of several ways:</p>
<ul><li><p><code>-j</code> followed by a number <br><br />
<code>-j2</code> means that up to 2 jobs can run in parallel.</p></li>
<li><p><code>-j+</code> followed by a number <br><br />
<code>-j+2</code> means that the maximum number of jobs is the number of cores plus 2.</p></li>
<li><p><code>-j-</code> followed by a number <br><br />
<code>-j-2</code> means that the maximum number of jobs is the number of cores minus 2.</p></li>
</ul>
<p>If you don't know how many cores the machine has, run the command below:</p>
<p><code>
$ parallel --number-of-cores <br>
8
</code></p>
<h3>Keeping output order</h3>
<p>Each job may output lines to the standard output. When multiple jobs are run in parallel, the default behavior is that a job's output is displayed as soon as the job finishes. You may find this confusing because the output order may be different from the input order. The <code>-k</code> option keeps the output sequence the same as the input sequence.</p>
<h3>Showing progress</h3>
<p>The <code>--eta</code> option reports progress while <code>GNU parallel</code> executes, including the estimated remaining time (in seconds).</p>
<h3>Input place-holder</h3>
<p><code>GNU parallel</code> substitutes the <code>{}</code> parameter with the next line in the standard input.</p><p>Each input line is a directory location, e.g., <code>/home/peter</code>. Instead of the full location, you can specify other parameters in order to extract a portion thereof - e.g., the directory name(<code>/home</code>) and the basename (<code>peter</code>). Please refer to the <a href="https://www.gnu.org/software/parallel/man.html#OPTIONS">man page</a> for details.</p>
<h2>Summary</h2>
<p><code>GNU parallel</code> is a tool that Linux administrators should add to their repertoire. Running a job in parallel can only improve one's efficiency. If you are already familiar with <code>xargs</code>, you will find the syntax familiar. Even if you are new to the command, there is a wealth of on-line help on the <a href="https://www.gnu.org/software/parallel/"><code>GNU parallel</code> website</a>.</p><div class="blogger-post-footer"><script type="text/javascript"><!--
google_ad_client = "pub-1640070864477148";
//728x90, created 11/11/07
google_ad_slot = "6009857287";
google_ad_width = 728;
google_ad_height = 90;
//--></script>
<script type="text/javascript"
src="http://pagead2.googlesyndication.com/pagead/show_ads.js">
</script></div>Peter Leunghttp://www.blogger.com/profile/05589860210899238688noreply@blogger.com1tag:blogger.com,1999:blog-8032022811235182759.post-73202397366739285252015-11-13T17:04:00.000-08:002018-02-18T20:20:06.383-08:00How to optimize Apache concurrency for running WordPress<div class="separator" style="clear: both; text-align: center;"><a href="https://4.bp.blogspot.com/-AjJ4YUk0sUg/VdNlQhe36HI/AAAAAAAACIE/55ZzyGFslUQ/s1600/people-309093_1280.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://4.bp.blogspot.com/-AjJ4YUk0sUg/VdNlQhe36HI/AAAAAAAACIE/55ZzyGFslUQ/s320/people-309093_1280.png" /></a></div>
<p>To a technical savvy person, hosting your own website on a <a href="https://en.wikipedia.org/wiki/Virtual_private_server">VPS</a> server can bring a tremendous sense of accomplishment and a wealth of learning opportunities. If <a href="https://wordpress.org/">WordPress</a> is what you fancy, with a minimal monthly financial commitment, you can host a WordPress website on the LAMP platform (<code><strong>L</strong>inux</code>, <code><strong>A</strong>pache</code>, <code><strong>M</strong>ySQL</code>, <code><strong>P</strong>HP</code>). For example, the entry-level, $5 per month plan offered by <a href="https://www.digitalocean.com/?refcode=fda198b040e5">DigitalOcean</a>, of which I am an affiliate, will give you a 512MB RAM, single-core VPS.</p>
<p>With such a small RAM capacity, you will need to optimize how your Apache webserver is configured to run PHP applications such as WordPress and Drupal. The goal is to maximize the number of concurrent web connections.</p>
<p>This tutorial details the Apache/PHP setup procedure on Debian 8.2, aka Jessie. The procedure assumes Apache is yet to be installed. However, if Apache2 is already installed, you will find practical information below on how to reconfigure Apache2 to run a different multi-processing module.</p>
<h2>Background knowledge</h2>
<p>According to a recent <a href="https://news.netcraft.com/archives/2015/10/16/october-2015-web-server-survey.html">Netcraft webserver survey</a>, Apache powers 46.91% of the top million busiest websites on the Internet. Busy websites mean many concurrent web connections.</p>
<p>Concurrent connection requests to Apache are handled by its <a href="https://httpd.apache.org/docs/2.4/mpm.html"><code><strong>M</strong></code>ulti-<code><strong>P</strong></code>rocessing <code><strong>M</strong></code>odules</a>. MPMs can be loosely classified as threaded or non-threaded. Older Apache releases default to a MPM named <a href="https://httpd.apache.org/docs/2.4/mod/prefork.html">Prefork</a>. This MPM is non-threaded. Each connection request is handled by a dedicated, self-contained Apache process.</p>
<p>Newer Apache releases default to a threaded MPM, either <a href="https://httpd.apache.org/docs/2.4/mod/worker.html">Worker</a> or <a href="https://httpd.apache.org/docs/2.4/mod/event.html">Event</a>. The Worker MPM uses one worker thread per connection. One issue with this approach is that a thread is tied up if the connection is kept alive despite it being inactive.</p>
<p>The Event MPM, a variant of Worker, addresses the aforesaid keep-alive issue. A main thread is used as the traffic controller that listens for requests and passes requests to worker threads on demand. In this scenario, an inactive but kept-alive connection does not tie up a worker thread.</p>
<p>Note that MPMs are mutually exclusive: only 1 MPM can be active at any given time.</p>
<p>Traditionally, the Apache core functionality serves static web content (e.g., HTML text and images). To serve dynamic content, such as PHP pages from WordPress, Apache requires special modules that execute PHP code.</p>
<p>For the Prefork MPM, each spawned Apache process embeds its own copy of the PHP handler (<code>mod_php</code>). Concurrency in this model is limited by the number of processes that Apache can spawn given the available memory.</p>
<p>For both Worker and Event MPMs, PHP requests are passed to an external FastCGI process, <a href="https://php-fpm.org/about/"><code>PHP5-FPM</code></a>. <code>PHP-FPM</code> stands for PHP-FastCGI Process Manager. Essentially, the webserver and the PHP handler are split to separate processes. Apache communicates with <code>PHP-FPM</code> through an Apache module, either <code>mod_fastcgi</code> or <code>mod_fcgid</code>. Optimizing concurrency in this model means configuring both the MPM and the PHP handler (PHP-FPM) to have pools of processes and threads to handle requests.</p>
<p>The rest of this tutorial covers the cases of installing the Event MPM from scratch as well as migrating to Event from the Prefork MPM. </p>
<h2>Installing Apache2</h2>
<p>This tutorial starts with the installation of Apache2. If Apache is already installed, you should find out which MPM is currently running using the command <code>apache2ctl -V</code>, and proceed to the next section. </p>
<p><code>
$ sudo apt-get update && sudo apt-get upgrade <br>
$ sudo apt-get install apache2
</code></p>
<p>Next, note the Apache version you just installed and the MPM that is loaded.</p>
<p><code>
$ sudo apache2ctl -V <br>
Server version: Apache/2.4.10 (Debian) <br>
... <br>
Architecture: 64-bit <br>
Server MPM: event <br>
threaded: yes (fixed thread count) <br>
forked: yes (variable process count) <br>
...
</code></p>
<p>The above output tells us that we are running Apache release 2.4. Beginning with 2.4, Apache runs the <a href="https://httpd.apache.org/docs/current/mod/event.html">Event MPM</a> by default.
If you are running an older version of Apache, the default MPM is either Prefork or Worker.</p>
<h2>Configuring Apache2</h2>
<ol>
<li><p>Back up the Apache configuration file, <code>/etc/apache2/apache2.conf.</code></p>
<p><code>
$ sudo cp /etc/apache2/apache2.conf{,.orig}
</code></p></li>
<li><p>Edit the configuration file.</p>
<p>Below is a subset of configuration parameters belonging to the Apache core module. You should adjust their values in order to optimize concurrency. The corresponding values are what I use for an entry-level VPS. </p>
<p><code>
Timeout 100 <br>
KeepAlive On <br>
MaxKeepAliveRequests 1000 <br>
KeepAliveTimeout 5
</code></p>
<p>For an in-depth explanation of the above parameters, please refer to <a href="https://httpd.apache.org/docs/current/mod/core.html">Apache on-line documentation</a>.</p></li>
<li><p>Enable <code>mod_rewrite</code>.</p>
<p>While the <a href="https://httpd.apache.org/docs/current/mod/mod_rewrite.html"><code>mod_rewrite</code></a> module is not strictly relevant to optimizing concurrency, I've included it here as a reminder to install the module. It is an important module for running WordPress.</p>
<p><code>
$ sudo a2enmod rewrite
</code></p></li>
</ol>
<h2>Installing Event MPM</h2>
<p>If you are already running the Event MPM, skip to the next section, 'Configuring Event MPM'. Otherwise, follow the procedure below.</p>
<ol>
<li><p>Install the Event MPM.</p>
<p><code>
$ sudo apt-get install apache2-mpm-event
</code></p></li>
<li><p>Disable existing MPM.</p>
<p>Recall that only 1 of Prefork, Worker or Event MPM can be running at any given time. Therefore, if you were previously running Prefork or Worker, you must first disable it, and then enable Event.</p>
<p>To disable the Prefork MPM, run this command: </p>
<p><code>
$ sudo a2dismod mpm_prefork
</code></p>
<p>To disable the Worker MPM, run this: </p>
<p><code>
$ sudo a2dismod mpm_worker
</code></p></li>
<li><p>Enable the Event MPM.</p>
<p><code>
$ sudo a2enmod mpm_event
</code></p>
<p>Note that the above enable and disable commands are quite 'forgiving'. If you attempt to enable an MPM that is already enabled, or disable an MPM that is already disabled, it will simply return a harmless informational message.</p></li>
</ol>
<h2>Configuring Event MPM</h2>
<p>To configure the Event MPM, modify its configuration file, <code>/etc/apache2/mods-available/mpm_event.conf</code>.
Before making any changes, back it up using the following command:</p>
<p><code>
$ sudo cp /etc/apache2/mods-available/mpm_event.conf{,.orig}
</code></p>
<p>Edit the file to specify the following parameters:</p>
<pre><code>
<IfModule mpm_event_module>
StartServers 2
MinSpareThreads 25
MaxSpareThreads 75
ThreadLimit 25
ThreadsPerChild 25
MaxRequestWorkers 250
MaxConnectionsPerChild 10000
ServerLimit 12
</IfModule>
</code></pre>
<p>The above configuration is what I recommend for an entry-level VPS (512MB RAM, single-core). You need to adjust the parameters to satisfy your own system requirements. For a detailed explanation of the above parameters, <a href="https://httpd.apache.org/docs/current/mod/worker.html">click here</a>. Note that the Event MPM shares the same parameters as the Worker MPM.</p>
<h2>Installing PHP5 handler</h2>
<p>To execute PHP code, Apache requires a PHP handler. <a href="https://php-fpm.org/about/"><code>PHP5-FPM</code></a> is the PHP handler to use with the Event MPM.</p>
<p>For a new PHP installation, install <code>php5-fpm</code> followed by the meta-package <code>php5</code>.</p>
<p><code>
$ sudo apt-get install php5-fpm php5
</code></p>
<p>In addition to the above packages, I also installed other PHP5 packages which WordPress requires.
While they are not strictly relevant to optimizing concurrency, I've included them here for completeness.</p>
<p><code>
$ sudo apt-get install php5-mysql php5-gd php5-curl
</code></p>
<h2>Configuring virtual host</h2>
<p>Suppose your WordPress website has the domain name <code>example.com</code>. To set up a virtual host with that domain name, follow the steps below:</p>
<ol>
<li><p>Create the Apache configuration file for <code>example.com</code>.</p>
<p>Instead of creating the file from scratch, use the default site as a template.</p>
<p><code>
$ sudo cp /etc/apache2/sites-available/000-default.conf /etc/apache2/sites-available/example.com.conf
</code></p></li>
<li><p>Edit the configuration file.</p>
<p>Customize the following site-specific parameters:</p>
<pre><code>ServerName example.com
ServerAlias www.example.com
ServerAdmin info@example.com
DocumentRoot /var/www/example.com
</code></pre></li>
<li><p>Create <code>DocumentRoot</code> directory.</p>
<p><code>
$ sudo mkdir /var/www/example.com <br>
$ sudo chown -R <webuser>:<webuser> /var/www/example.com
</code></p>
<p>Notes: </p>
<ul>
<li><p>WordPress should be installed in the directory <code>/var/www/example.com</code>. For instructions on how to install WordPress, refer to my <a href="https://linuxcommando.blogspot.com/2013/12/install-wordpress-38-on-debian-wheezy.html">earlier post</a>.</p></li>
<li><p>The <code>DocumentRoot</code> directory should be owned by a non-root user.</p></li>
</ul></li>
<li><p>Enable the new site.</p>
<p><code>
$ sudo a2ensite example.com.conf
</code></p></li>
<li><p>Disable the default site.</p>
<p><code>
$ sudo a2dissite 000-default.conf
</code></p></li>
</ol>
<h2>Configuring PHP handler</h2>
<p>Follow the procedure below to configure PHP5-FPM.</p>
<ol>
<li><p>Create a custom PHP configuration file for <code>example.com</code> by copying the template from the default site.</p>
<p><code>
$ sudo cp /etc/php5/fpm/pool.d/www.conf /etc/php5/fpm/pool.d/example.com.conf
</code></p></li>
<li><p>Edit the configuration file.</p>
<p>Customize the following parameters.</p>
<p><code>
[example.com] <br>
user = <webuser> <br>
group = <webuser> <br>
listen = /var/run/php5-fpm_example.com.sock <br>
pm = dynamic <br>
pm.max_children = 5 <br>
pm.start_servers = 2 <br>
pm.min_spare_servers = 1 <br>
pm.max_spare_servers = 3 <br>
pm.max_requests = 2000
</code></p>
<p>Notes:</p>
<ul>
<li><p>The <code>user</code> and <code>group</code> parameters specify respectively the Unix user and group names under which the FPM processes will run. You should specify a non-root user for both.</p></li>
<li><p>The <code>listen</code> parameter specifies the source address that the FPM will listen to for receiving PHP requests. In this case, it will listen to the Unix socket <code>/var/run/php5-fpm_example.com.sock</code>.</p></li>
<li><p>The rest of the parameters are for an entry-level VPS system. You should adjust their values to satisfy your system requirements.</p></li>
<li><p><a href="https://php.net/manual/en/install.fpm.configuration.php">Click here</a> for more details about the above parameters.</p></li>
</ul></li>
<li><p>Restart <code>PHP5-FPM</code>.</p>
<p><code>
$ sudo systemctl restart php5-fpm
</code></p></li>
</ol>
<h2>Installing FastCGI</h2>
<p>Apache requires a FastCGI module to interface with the external <code>PHP5-FPM</code> processes.
You can use 1 of 2 FastCGI modules: <code>mod_fastcgi</code> and <code>mod_fcgid</code>.
<a href="https://www.apachelounge.com/viewtopic.php?t=4385">Click here</a> for a discussion of their differences.
This tutorial uses <a href="https://fastcgi-archives.github.io/mod_fastcgi.html">mod_fastcgi</a>.</p>
<p>Before you install <code>mod_fastcgi</code>, you must:</p>
<ol>
<li><p>Enable <code>non-free</code>. </p>
<p>Debian pre-packages the <code>mod_fastcgi</code> module in the <code>non-free</code> archive area of its repositories. Make sure that <code>non-free</code> is included in the <code>/etc/apt/sources.list</code> file.</p></li>
<li><p>Disable <code>mod_php</code>. </p>
<p>If Apache2 was previously installed with the Prefork MPM, most likely, it is configured to execute PHP using the <code>mod_php</code> module. In this case, you must disable the <code>mod-php</code> module before you install <code>mod_fastcgi</code>. Otherwise, the install will fail with the error message, 'Apache is running a threaded MPM, but your PHP Module is not compiled to be threadsafe. You need to recompile PHP.'</p>
<p>To disable <code>mod_php</code>, run this command: </p>
<p><code>
$ sudo a2dismod php5
</code></p></li>
</ol>
<p>To install <code>mod_fastcgi</code>, execute the following command:</p>
<p><code>
$ sudo apt-get install libapache2-mod-fastcgi
</code></p>
<h2>Configuring FastCGI</h2>
<ol>
<li><p>Back up configuration file.</p>
<p>Before you edit the configuration file <code>/etc/apache2/mods-available/fastcgi.conf</code>, back it up using the following command.</p>
<p><code>
$ sudo cp /etc/apache2/mods-available/fastcgi.conf{,.orig}
</code></p></li>
<li><p>Edit the file.</p>
<p>Insert the following lines:</p>
<p><pre><code>
<IfModule mod_fastcgi.c>
AddHandler php5-fcgi .php
Action php5-fcgi /php5-fcgi
Alias /php5-fcgi /usr/lib/cgi-bin/php5-fcgi_example.com
FastCgiExternalServer /usr/lib/cgi-bin/php5-fcgi_example.com -socket /var/run/php5-fpm_example.com.sock -pass-header Authorization
<Directory /usr/lib/cgi-bin>
Require all granted
</Directory>
</IfModule>
</pre></code></p></p>
<p>Notes:</p>
<ul>
<li><p><code>example.com</code> should be replaced with your own domain name.</p></li>
<li><p>To access the website, you need to grant the proper permission explicitly using the <code>Require all granted</code> statement. Without it, access to the website will be denied with the error message 'You don't have permission to access /php5-fcgi/index.php on this server.'</p></li>
</ul></li>
<li><p>Enable additional modules.</p>
<p><code>
$ sudo a2enmod actions fastcgi alias
</code></p></li>
<li><p>Restart Apache.</p>
<p>The final step is to restart Apache to make all the above changes go live.</p>
<p><code>
$ sudo systemctl restart apache2
</code></p></li>
</ol>
<h2>Threads in action</h2>
<p>Concurrency for WordPress occurs at both the webserver (Apache2) and the PHP handler (PHP-FPM) levels. You can use the <code>ps -efL</code> command to monitor the processes and threads at either level.</p>
<p>To monitor Apache processes and threads, execute the following <code>ps</code> command.</p>
<p><code>
$ ps -efL |grep apach[e] <br>
www-data 31681 24441 31681 0 27 03:25 ? 00:00:00 /usr/sbin/apache2 -k start <br>
www-data 31681 24441 31684 0 27 03:25 ? 00:00:00 /usr/sbin/apache2 -k start <br>
...
</code></p>
<p>The second and the fourth columns are the process ID (PID) and the thread ID respectively. Note that the above output reports 2 different threads (31681 and 31684) of the same process (31681).</p>
<p>Execute the following command to monitor PHP.</p>
<p><code>
$ ps -efL |grep ph[p] <br>
root 24398 1 24398 0 1 Nov10 ? 00:00:17 php-fpm: master process (/etc/php5/fpm/php-fpm.conf) <br>
peter 31519 24398 31519 0 1 03:14 ? 00:00:17 php-fpm: pool example.com <br>
peter 31520 24398 31520 0 1 03:14 ? 00:00:16 php-fpm: pool example.com <br>
peter 31827 24398 31827 0 1 04:15 ? 00:00:15 php-fpm: pool example.com
</code></p>
<h2>Conclusion</h2>
<p>When traffic to your website increases over time, your webserver must scale up to handle the increase in traffic. This tutorial explains how to configure Apache2 and PHP to optimize the number of concurrent connections. After you try it out, if you still find that your website cannot keep up with the traffic, you should consider upgrading your VPS plan to have more RAM.</p>
<p>If you are interested in WordPress, please refer to my earlier posts.</p>
<ul>
<li><a href="https://linuxcommando.blogspot.com/2015/10/wpscan-stop-feeling-vulnerable-about.html">WPScan-scanning for vulnerabilities</a></li>
<li><a href="https://linuxcommando.blogspot.com/2013/12/install-wordpress-38-on-debian-wheezy.html">How to install WordPress on Debian</a></li>
</ul>
<div class="blogger-post-footer"><script type="text/javascript"><!--
google_ad_client = "pub-1640070864477148";
//728x90, created 11/11/07
google_ad_slot = "6009857287";
google_ad_width = 728;
google_ad_height = 90;
//--></script>
<script type="text/javascript"
src="http://pagead2.googlesyndication.com/pagead/show_ads.js">
</script></div>Peter Leunghttp://www.blogger.com/profile/05589860210899238688noreply@blogger.com0tag:blogger.com,1999:blog-8032022811235182759.post-4144232376110852752015-10-30T16:45:00.000-07:002018-02-18T19:54:53.317-08:00WPScan: Stop feeling vulnerable about WordPress<div class="separator" style="clear: both; text-align: center;"><a href="https://4.bp.blogspot.com/-Qqskw4hETZ8/VjJrWlT2xTI/AAAAAAAACI0/imNAOUaZJvc/s1600/wordpress-588494_1280.jpg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://4.bp.blogspot.com/-Qqskw4hETZ8/VjJrWlT2xTI/AAAAAAAACI0/imNAOUaZJvc/s320/wordpress-588494_1280.jpg" /></a></div>
<h2>Introduction</h2>
<p><a href="https://wordpress.org/">WordPress</a> is the most popular <a href="https://en.wikipedia.org/wiki/Content_management_system">content management system (CMS)</a> on the planet today.
You can customize the look and feel of a WordPress website using third-party themes.
If you want a functionality not offered by the WordPress core, you will most likely find a third-party plugin that satisfies your requirement. With the plethora of themes and plugins comes a major challenge in assuring their quality. Intruders can potentially exploit the vulnerabilities in poorly designed themes and plugins to gain unauthorized access to a WordPress website.</p>
<p><a href="https://wpscan.org/">WPScan</a> is a WordPress vulnerability scanner that is free for non-commerical use.
It scans your WordPress website and reveals any known vulnerabilities in the installed plugins and themes.</p>
<p>The rest of this post explains how to install and run WPScan.</p>
<h2>Installation</h2>
<p>WPScan comes pre-installed on only a handful of lesser-known Linux distributions. If you run Debian, Ubuntu, Centos or Fedora, you must download the WPScan source and build it yourself. Because WPScan is written in Ruby, to build WPScan, you need to install the Ruby development environment. </p>
<p>Your first decision is to select a machine on which to build WPScan. This is the machine you use to launch WPScan later. Note that you can (and should) run WPScan on a different machine than the WordPress host. The examples in this post are based on a Debian 8.2 machine, aka Jessie.</p>
<p>Your next decision is how you will install the Ruby development environment, including the supporting development libraries.
The WPScan website outlines 2 ways to install the necessary development environment on a Debian server: the Debian package management system and the Ruby Version Manager(RVM).</p>
<p>RVM is the WPScan-recommended method. It allows you to install multiple, self-contained Ruby environments on the same system.
RVM puts a dedicated Ruby environment under your <code>Home</code> directory (e.g., <code>/home/peter/.rvm</code>). You can find the RVM procedure on the <a href="https://wpscan.org/">WPScan home page</a>. I've followed the steps, and it works as advertised. </p>
<p>I opted instead for the Debian package manager method because it is a shorter procedure and I did not need the versatility (and the complexity) that RVM offers.</p>
<p>Below are the steps to install WPScan using the Debian package manager. The procedure is largely based on what is on the WPScan home page. I've added a couple of missing packages that are actually required.</p>
<ol>
<li><p>Update your Debian repository.</p>
<p><code>
$ sudo apt-get update
</code></p></li>
<li><p>Install the supporting development packages.</p>
<p><code>
$ sudo apt-get install git make gcc ruby ruby-dev libcurl4-openssl-dev zlib1g-dev
</code></p></li>
<li><p>Download the WPScan source.</p>
<p><code>
$ cd; git clone https://github.com/wpscanteam/wpscan.git
</code></p></li>
<li><p>Install WPScan.</p>
<p><code>
$ cd wpscan <br>
$ sudo gem install bundler <br>
$ bundle install --without test --path vendor/bundle
</code></p></li>
</ol>
<p><a href="https://3.bp.blogspot.com/-F233IDDr3CU/VjOur0VEWGI/AAAAAAAACJY/cbb2nNJf3EU/s1600/tick-308750_640.png" imageanchor="1" ><img border="0" src="https://3.bp.blogspot.com/-F233IDDr3CU/VjOur0VEWGI/AAAAAAAACJY/cbb2nNJf3EU/s200/tick-308750_640.png" /></a></p>
<h2>Operation</h2>
<p>The WPScan development team maintains a <a href="https://wpvulndb.com">WordPress vulnerability database</a>.
Before you scan your WordPress website, you should first update the local copy of the vulnerability database as follows: </p>
<p><code>
$ cd $HOME/wpscan <br>
$ ruby wpscan.rb --update
</code></p>
<p>To scan a WordPress website for vulnerabilities, use the following general WPScan command sequence.</p>
<p><code>
$ cd $HOME/wpscan <br>
$ ruby wpscan.rb --url http://yourWP.com --enumerate <options>
</code></p>
<p>Notes:</p>
<ul>
<li><p>The <code>--url</code> parameter specifies which WordPress website to scan.</p></li>
<li><p>The <code>--enumerate</code> parameter specifies which WordPress components to scan for vulnerabilities.
I use the following options: <br>
* <code>vp</code>: Scan only vulnerable plugins. <br>
* <code>vt</code>: Scan only vulnerable themes. <br>
* <code>u</code>: Scan users. <br>
* <code>tt</code>: Scan timthumbs. <br>
You can combine multiple options with separating commas. <br>
For example, <code>--enumerate vp,vt,u</code> </p></li>
<li><p>For further on-line help, run this command:</p>
<p><code>
$ ruby wpscan.rb --help
</code></p></li>
</ul>
<h2>Results Analysis</h2>
<p>I invoked WPScan to scan the WordPress website deployed on my desktop (<code>localhost</code>). The report is displayed below. Note that the output has been abbreviated in order to improve its overall readability. </p>
<p>WPScan managed to find:</p>
<ol>
<li><p>3 vulnerabilities in the WordPress core.</p>
<p>Those vulnerabilities were fixed in the most recently released version of WordPress.</p></li>
<li><p>1 vulnerability in the WP Super Cache plugin.</p>
<p>Again, the vulnerability was fixed in the most recently released version of the plugin.</p></li>
<li><p>No known vulnerability in the installed themes.</p></li>
</ol>
<p><font size="1"></p>
<p><pre><code>
$ ruby wpscan.rb --url localhost --enumerate vp,vt
---
[+] WordPress version 4.3 identified from meta generator
[!] 3 vulnerabilities identified from the version number
[!] Title: WordPress <= 4.3-Authenticated Shortcode Tags Cross-Site Scripting (XSS)
Reference: https://wpvulndb.com/vulnerabilities/8186
[i] Fixed in: 4.3.1
[!] Title: WordPress <= 4.3-User List Table Cross-Site Scripting (XSS)
Reference: https://wpvulndb.com/vulnerabilities/8187
[i] Fixed in: 4.3.1
[!] Title: WordPress <= 4.3-Publish Post and Mark as Sticky Permission Issue
Reference: https://wpvulndb.com/vulnerabilities/8188
[i] Fixed in: 4.3.1
---
[+] Enumerating installed plugins (only ones with known vulnerabilities)...
[+] We found 1 plugins:
[+] Name: wp-super-cache - v1.4.4
[!] The version is out of date, the latest version is 1.4.6
[!] Title: WP Super Cache <= 1.4.4 - Cross-Site Scripting (XSS)
Reference: https://wpvulndb.com/vulnerabilities/8197
[i] Fixed in: 1.4.5
[+] Enumerating installed themes (only ones with known vulnerabilities)...
[+] No themes found
</code></pre></p>
<p></font></p>
<h2>Conclusion</h2>
<p>WPScan is an important tool in your defense against possible attacks on your WordPress websites. It is recommended that you schedule WPScan to run regularly to detect known WordPress vulnerabilities. Yet, running WPScan is only half of your job. You remain vulnerable until you patch the vulnerabilities.</p>
<p>In general, the WordPress community fixes most known vulnerabilities and distributes the fixes quickly after the vulnerabilities are first reported. It is important that you keep your WordPress core and the third-party themes and plugins up-to-date. If your WordPress platform is up-to-date, WPScan will most likely return a clean report, and you can stop feeling vulnerable about your WordPress website.</p>
<p> Below are my other posts on WordPress:
<ul>
<li> <a href="https://linuxcommando.blogspot.com/2015/11/how-to-optimize-apache-concurrency-for.html">Optimizing WordPress concurrency</a> </li>
<li> <a href="https://linuxcommando.blogspot.com/2013/12/install-wordpress-38-on-debian-wheezy.html">How to install WordPress on Debian</a>
</ul><div class="blogger-post-footer"><script type="text/javascript"><!--
google_ad_client = "pub-1640070864477148";
//728x90, created 11/11/07
google_ad_slot = "6009857287";
google_ad_width = 728;
google_ad_height = 90;
//--></script>
<script type="text/javascript"
src="http://pagead2.googlesyndication.com/pagead/show_ads.js">
</script></div>Peter Leunghttp://www.blogger.com/profile/05589860210899238688noreply@blogger.com0tag:blogger.com,1999:blog-8032022811235182759.post-54788882878896459302015-07-29T20:29:00.001-07:002018-02-18T18:49:04.768-08:00MySQLTuner series part 3 - minimizing MySQL memory footprint<div class="separator" style="clear: both; text-align: center;"><a href="https://3.bp.blogspot.com/-b5XfN7V9REo/VbZjjFECETI/AAAAAAAACG4/8obDlQxboq8/s1600/memory-332492_1280.jpg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://3.bp.blogspot.com/-b5XfN7V9REo/VbZjjFECETI/AAAAAAAACG4/8obDlQxboq8/s320/memory-332492_1280.jpg" /></a></div>
<p>This is part 3 of the series on using <a href="https://github.com/major/MySQLTuner-perl"><code>MySQLTuner</code></a> to optimize MySQL database performance and stability. <a href="https://linuxcommando.blogspot.com/2015/06/improving-mysql-stability-performance.html">Part 1</a> explains how to install and run <code>MySQLTuner</code>. <a href="https://linuxcommando.blogspot.com/2015/07/mysqltuner-series-part-2-defragmenting.html">Part 2</a> addresses the area of database defragmentation. This post illustrates how to manage <code>MySQL</code> memory footprint.</p>
<h2><code>MySQLTuner</code> output</h2>
<p><code>MySQLTuner</code> was used to analyze a WordPress database deployed on the <strong>LAMP</strong> platform (<code><strong>L</strong>inux</code>, <code><strong>A</strong>pache</code>, <code><strong>M</strong>ySQL</code>, <code><strong>P</strong>HP</code>). The host was a <a href="https://en.wikipedia.org/wiki/Virtual_private_server">VPS</a> server with only 512 MB of memory.</p>
<p><code>
$ perl mysqltuner.pl
</code></p>
<p><a href="https://3.bp.blogspot.com/-hqrqn4qUu84/VYR9VlQba8I/AAAAAAAACE8/sRus1I9Maoo/s1600/mysqltuner_output.png" imageanchor="1" ><img border="0" src="https://3.bp.blogspot.com/-hqrqn4qUu84/VYR9VlQba8I/AAAAAAAACE8/sRus1I9Maoo/s640/mysqltuner_output.png" /></a></p>
<p>If you scroll down to the <code>Recommendations</code> section of the above report, it is hard to miss the eye-catching message:
'MySQL's maximum memory usage is dangerously high. Add RAM before increasing MySQL buffer variables.'</p>
<p>Indeed, adding more RAM is often the cheapest and simplest solution to out-of-memory problems. By spending an extra $5 per month, I can upgrade my VPS server to have 1 GB of RAM. But, before you go spend your hard-earned money on RAM, let's explore some other ways to reduce MySQL's memory footprint.</p>
<h2>Maximum number of database connections</h2>
<p>Lines that begin with two exclamation marks ('!!') are warnings. Note the following lines in the above <code>Performance Metrics</code> section: </p>
<p><code>
-------- Performance Metrics -------------------------------------- <br>
... <br>
[--] Up for: 36d 23h 28m 39s (758K q [0.237 qps], 53K conn, TX: 6B, RX: 98M) <br>
[--] Total buffers: 192.0M global + 2.7M per thread (151 max threads) <br>
[!!] Maximum possible memory usage: 597.8M (120% of installed RAM) <br>
... <br>
[OK] Highest usage of available connections: 8% (13/151)
</code></p>
<div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-GiP5MBoMTio/Vbf1BvGrzmI/AAAAAAAACHY/KpNX4yjfDkU/s1600/ethernet-490027_1280.jpg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://1.bp.blogspot.com/-GiP5MBoMTio/Vbf1BvGrzmI/AAAAAAAACHY/KpNX4yjfDkU/s320/ethernet-490027_1280.jpg" /></a></div>
<p>According to the above warning, <code>MySQL</code> could potentially use up to 597.8 MB of RAM. Where did the number come from?</p>
<p>The number was derived from the preceding line. MySQL required 192MB globally and 2.7 MB per connection to the database. By default, the maximum number of connections was 150+1.
(The 151<sup>st</sup> connection would be restricted to database administration only.) Hence, the maximum memory usage was 192 + 150 * 2.7, equaling 597.</p>
<p>Should you allow for 150 connections? Keep in mind that each connection, even in the idle state, will take up some memory.
<code>MySQLTuner</code> can help you answer the question with confidence.</p>
<p><code>MySQLTuner</code> reports the highest number of concurrent connections since the last <code>MySQL</code> restart (13 in the above example). The database should be up for a minimum of 24 hours before you run <code>MySQLTuner</code>. In fact, the longer the time elapses since the last restart, the more trustworthy is the statistic.</p>
<p> You can find out from the <code>MySQLTuner</code> report how long <code>MySQL</code> has been up. Go back to the first line under the <code>Performance Metrics </code> section. In the above example, <code>MySQL</code> had been up for 36 days since the last restart.</p>
<p>Although <code>MySQL</code> was configured for accepting 150 connections, the highest number of concurrent connections made in the past 36 days was only 13 (8% of the maximum). In light of that knowledge, we could lower the maximum number of connections allowed, therefore, reducing the total memory footprint for <code>MySQL</code>.</p>
<p>Before we go ahead to reconfigure <code>MySQL</code>, we will consider the wait-timeout threshold which affects how long idle connections stay alive before timing out.</p>
<h2>Wait timeout</h2>
<div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-KcSYCRBmlOs/Vbfw5yIGIAI/AAAAAAAACHM/CCl0dAlKvKA/s1600/the-eleventh-hour-758726_1280.jpg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://1.bp.blogspot.com/-KcSYCRBmlOs/Vbfw5yIGIAI/AAAAAAAACHM/CCl0dAlKvKA/s320/the-eleventh-hour-758726_1280.jpg" /></a></div>
<p>One of the <code>General recommendations</code> in the above example was:</p>
<p>'Your applications are not closing MySQL connections properly.'</p>
<p>In other words, database connections were opened but not properly closed after queries or updates were already completed.
These idle connections would hang around until a predefined timeout threshold was reached. The default timeout threshold was 8 hours. So, if a query completed in 2 seconds, but because the connection was not closed properly, the connection would live for another 28,798 seconds before timing out. In the meantime, the idle connections continued to consume resources including counting toward the maximum number of open connections.</p>
<p>The culprit was easily identified in the above case: the database was used exclusively for WordPress, an application written in PHP. However, solving the problem can be out of your reach, unless you are a PHP developer.</p>
<p>The good news is that you can reduce the timeout interval by adjusting a <code>MySQL</code> configuration parameter. By making idle connections time out faster, there will be less concurrent connections. For WordPress/PHP applications, I set the wait timeout to be 60 seconds. </p>
<p>It is also worth mentioning that because there are less idle connections due to quicker timeout, you can further reduce the maximum number of connections.</p>
<h2>Re-configuring <code>MySQL</code></h2>
<p>To change the maximum number of connections or the wait timeout threshold, edit the <code>MySQL</code> configuration file as follows.</p>
<p><code>
$ sudo vi /etc/mysql/my.cnf
</code></p>
<p>The configuration variables of interest are <code>max_connections</code>, and <code>wait_timeout</code>. Enter a value for each variable using the following syntax:</p>
<p><code>
max_connections = 50 <br>
wait_timeout = 60
</code></p>
<p>For the above configuration changes to take effect, a restart of the <code>MySQL</code> daemon is needed.</p>
<p> For non-systemd systems, run the following command: </p>
<p><code>
$ sudo service mysql restart <br>
</code></p>
<p> For systemd-enabled systems, run:</p>
<p><code>
$ sudo systemctl restart mysql <br>
</code></p>
<p>Alternatively, you can dynamically change the configuration variables, thereby avoiding the database restart. To do that, issue the following commands. </p>
<p><code>
$ mysql -u root -p -e "SET GLOBAL max_connections=50;" <br>
$ mysql -u root -p -e "SET GLOBAL wait_timeout=60;"
</code></p>
<p>Note that modifying the <code>MySQL</code> configuration file is still required if you want the changes to persist after future system restarts.</p>
<h2>What's next?</h2>
<p><code>MySQLTuner</code> is not something you run once and forget about it. Your web traffic pattern changes over time. You should schedule to run it regularly and examine the output. Please refer back to <a href="https://linuxcommando.blogspot.com/2015/06/improving-mysql-stability-performance.html">Part 1</a> of this series for instructions on how to schedule a run.</p>
<p>The more knowledgeable you are about database optimization, the more effective you become on using the information provided by <code>MySQLTuner</code>. I recommend the following videos if you want to learn more about <code>MySQL</code> optimization:</p>
<ul>
<li><p><a href="https://www.youtube.com/watch?v=u70mkgDnDdU&utm_content=bufferc2f4f&utm_medium=social&utm_source=twitter.com&utm_campaign=buffer">Performance Tuning Best Practices for MySQL</a></p></li>
<li><p><a href="https://www.percona.com/resources/mysql-videos/mysql-56-configuration-optimization?utm_content=buffer53c22&utm_medium=social&utm_source=twitter.com&utm_campaign=buffer">MySQL 5.6 Configuration Optimization</a></p></li>
</ul>
<p>You may also be interested in the following MySQL-related articles from this blog:</p>
<ul>
<li><p><a href="https://linuxcommando.blogspot.com/2015/02/how-to-update-mysql-table-with-data.html">How to update a MySQL table with data from itself</a></p></li>
<li><p><a href="https://linuxcommando.blogspot.com/2013/03/deciphering-mysql-error-codes.html">Looking up the meaning of MySQL error codes</a></p></li>
<li><p><a href="https://linuxcommando.blogspot.com/2015/04/configuring-monit-free-system.html">Configuring Monit to monitor database health</a></p></li>
</ul>
For Apache optimization, please refer to <a href="https://linuxcommando.blogspot.com/2015/11/how-to-optimize-apache-concurrency-for.html">this blog post</a>.<div class="blogger-post-footer"><script type="text/javascript"><!--
google_ad_client = "pub-1640070864477148";
//728x90, created 11/11/07
google_ad_slot = "6009857287";
google_ad_width = 728;
google_ad_height = 90;
//--></script>
<script type="text/javascript"
src="http://pagead2.googlesyndication.com/pagead/show_ads.js">
</script></div>Peter Leunghttp://www.blogger.com/profile/05589860210899238688noreply@blogger.com1