Tuesday, April 3, 2018

Scanning HTTPS for Mixed Content

Back in 2014, Google raised the awareness of using HTTPS ("Secure HTTP") by making its use a ranking signal in Google search algorithms. HTTPS essentially establishes secure encrypted connections to the cloud. Google further raised the stake of not using HTTPS by announcing that, beginning in July 2018, the Google Chrome browser with the release of Chrome 68 will mark all HTTP websites as being insecure. The consequence of not converting to HTTPS is that site visitors will be persuaded by the warning message to bounce from your website.
Even before the impending drop dead date, Chrome and other popular web browsers such as Firefox and Edge have been warning visitors to HTTP-connected sites with an informational message.


Web administrators had taken heed and converted their websites to HTTPS, many taking advantage of the free SSL certificates issued by Let's Encrypt. However, if you have successfully converted to HTTPS, your work may not be done. You still need to verify that your website is properly recognized as being secure. You want to see the padlock icon displayed next to the web page's URL in the browser window.

To many administrators' surprise, even a properly converted HTTPS website may still be marked as being insecure. This is most likely due to the website's mixed content. For a web page to be deemed secure, everything loaded by that page must be encrypted by HTTPS. A web page with mixed content loads both encrypted as well as non-encrypted contents such as images, videos, stylesheets and scripts.

While it is possible to manually spot mixed web content on a web page, checking a non-trivial website requires automation. Mixed Content Scan is a command-line web crawler which scans for mixed content. The rest of this post explains how to install and use the tool.

Installation

Mixed Content Scan is a batch PHP application. To install the tool, use composer, a PHP package dependency manager. For the latest instructions on how to install composer, please refer to this link. Note that the said procedure installs composer in the current directory. Optionally, move the executable to a globally accessible directory using the following command.
$ sudo mv composer.phar /usr/local/bin/composer
To install Mixed Content Scan:
$ composer global require bramus/mixed-content-scan:~2.8
The Mixed Content Scan executable is placed in ~/.config/composer/vendor/bramus/mixed-content-scan/bin.

Scanning for mixed content

To scan a website for mixed content, simply provide its URL as an argument to Mixed Content Scan:
$ cd ~/.config/composer/vendor/bramus/mixed-content-scan/bin
$ ./mixed-content-scan https://shadowofyourwings.com/
By default, the tool outputs the scan report on the terminal("standard output"). Alternatively, you can specify an output file using the --output parameter as follows:
$ cd ~/.config/composer/vendor/bramus/mixed-content-scan/bin
$ ./mixed-content-scan --output <some/file/path> https://shadowofyourwings.com/
You can also use the --ignore parameter to specify a file which contains URL patterns that the tool will ignore and not scan. The example site I use is a WordPress website. The scanning tool comes with a sample ignore file for WordPress which is located in ~/.config/composer/vendor/bramus/mixed-content-scan/bin/ignorepatterns/wordpress.txt.


$ cd ~/.config/composer/vendor/bramus/mixed-content-scan/bin
$ ./mixed-content-scan --ignore=~/.config/composer/vendor/bramus/mixed-content-scan/bin/ignorepatterns/wordpress.txt https://shadowofyourwings.com/
[2018-02-16 16:53:18] MCS.NOTICE: Scanning https://shadowofyourwings.com/
[2018-02-16 16:53:18] MCS.ERROR: 00000 - https://shadowofyourwings.com/
[2018-02-16 16:53:18] MCS.WARNING: http://gmpg.org/xfn/11
[2018-02-16 16:53:19] MCS.ERROR: 00001 - https://shadowofyourwings.com/about
[2018-02-16 16:53:19] MCS.WARNING: http://shadowofyourwings.com/wp-content/uploads/2017/05/peterLeung.jpg
[2018-02-16 16:53:19] MCS.WARNING: http://gmpg.org/xfn/11

[2018-02-16 16:53:20] MCS.ERROR: 00002 - https://shadowofyourwings.com/contact
[2018-02-16 16:53:20] MCS.WARNING: http://gmpg.org/xfn/11
... <output snipped> ...
[2018-02-16 16:53:38] MCS.NOTICE: Scanned 26 pages for Mixed Content

Mixed Content Scan numbers each page scanned, starting from 00000. In the above example, the About page (00001) has been flagged as having mixed content. The sources of mixed content as loaded by that page are twofold:
  1. Vulnerable image file.
    The peterLeung.jpg file is being loaded via the insecure HTTP connection. The fix is simple: go to the WordPress administration web page, and change HTTP to HTTPS on the About web page.
  2. Theme header profile
    The header of the default twentyseventeen WordPress theme contains a reference to http://gmpg.org/xfn/11. The code is in <document root>/wp-content/themes/twentyseventeen/header.php.

    Although the scanner reports its occurrence as a violation, browsers generally do not flag this as a mixed content error. This error can be safely ignored.

Friday, March 16, 2018

A review of 3 best-of-breed Markdown editors


As a technology blogger, I write HTML documents that are hosted on different platforms such as WordPress, Drupal, and Blogger. I like to compose HTML using the Markdown markup language. Unfortunately, the HTML editors bundled with the aforementioned platforms do not support Markdown natively. It is true that you can download Markdown plugins for WordPress and Drupal. But, at the end, I still find the HTML editors to be too intrusive for a writer such as myself to stay focused and productive.

Fortunately, there are many good special-purpose Markdown editors out there. My web authoring process involves first composing the document using a Markdown editor, and then copying and pasting the output HTML into the Content Management System(CMS). Below, I evaluate 3 open-source Markdown editors: justmd, Remarkable, and ghostwriter.

I will evaluate each editor from two sometimes conflicting viewpoints, that of a geek and a writer. As a geek, I side with editors that have many bells and whistles. But, as a writer, I prefer editors that help me create, often by filtering out as much distraction as possible, and forcing me to focus on the next word, phrase, sentence to put on the page.

justmd

justmd is a minimalist, bare-bones Markdown editor. When you open justmd, you will see a single window with 2 window panes of equal size, located side-by-side. One pane is where you enter the Markdown text; the other is the HTML preview pane. Although you can change the overall size of the encompassing window, you cannot change the ratio of the 2 panes. The geek in me cannot help but cringe at the discovery. After all, it is common among Markdown editors (including Remarkable and ghostwriter) to have separate input and preview windows that you can independently resize and even hide. Conversely, the writer in me gives justmd a big shout-out for its austere simplicity. You just open the app, and immediately start writing, without having to adjust the size of any window component. Writers will find justmd more conducive to writing than many editors that are much more customizable.

Minimalist as it is designed to be, justmd, as a Markdown editor, is not feature complete in its current status. The following features, which I deem to be very important for writers, are still missing in justmd:

  • Spellchecker.
  • Word count.
  • Auto save.

This post was written entirely using justmd, and the overall experience was very positive. The lack of a spellchecker and word counter did not hamper the writing at all. On the contrary, it enhances my productivity by breaking the bad habits of constantly checking the word count and looking out for spelling errors in the midst of writing. Most Content Management Systems are capable of spellchecking and word counting. So, those tasks can be deferred until later, after you paste the HTML into the CMS.

Finally, I comment on the ease of installing justmd. None of the 3 editors being reviewed here are pre-packaged in the official repository of a major Linux distribution. Having said that, installing justmd is as easy as 1-2-3.

  1. Download compressed tarball from justmd website.
  2. Uncompress the tarball using command tar -zxvf justmd-linux-x64-v1.1.1.tar.gz.
  3. Create shortcut to justmd binary.

Remarkable

Featurewise, Remarkable is middle-of-the-road, between justmd and ghostwriter. It has word counting, but no spellchecking. Like justmd, both input and preview functions coexist as panes side-by-side in a single window, but you can stack them vertically or horizontally, and you can resize each pane proportionally within the window.

Now, as a writer, I find Remarkable's user interface too colorful, too distracting. Specifically, its overly generous use of color for syntax highlighting and icon design is detrimental to the primary writing task. With color, less is more.

You can download the Remarkable package in .deb or .rpm format from its Linux download page. Users of Debian, Ubuntu, Fedora, SUSE, and Arch systems will find installation straightforward.

ghostwriter

ghostwriter is the most mature and feature complete of all 3 MarkDown editors. It offers spellchecking, word counting, auto saves, and much more.

Two unique features are especially noteworthy to writers: Hemingway and Focus modes. In Hemingway mode, two particular keyboard keys are disabled, namely, the delete and the backspace keys. The rationale is to increase productivity by delaying document editing as much as possible. In Focus mode, only the portion of the document you are working on is made prominent, and the rest fades out. You can configure the focus to be the current sentence, the current single or 3 lines, or the current paragraph.

Despite the rich feature set, the ghostwriter user interface is surprising clean and uncluttered.

The input and live preview functions reside in separate windows that you can resize and move around individually. Keen observers will definitely notice there is a real-time lag between actual text input and the update of the live preview. This is not a bug in the program. On the contrary, ghostwriter is programmed to only update the live preview when you stop typing(for a fraction of a second). The technical reason given by the developers is that the delay smoothens the jitters in synchronizing the rendering of large files. I can see many writers actually support this design decision because attention should be primarily focused on the writing, not the rendering, of the document.

Recall that the overall objective for using a MarkDown editor is to generate HTML code to insert into a CMS. With justmd and Remarkable, you need to first export to a HTML file, and then import the file (or copy and paste its contents) into the CMS. On the other hand, ghostwriter provides a shortcut Copy HTML button which is discreetly tucked away at the bottom right of the window. The button is a minor feature in the overall design scheme, but has a disproportionally high value to end users. Clicking the button copies the HTML code in its entirety into the clipboard. Importing the HTML into the CMS simply involves pasting the contents of the clipboard.

ghostwriter provides packages for Ubuntu , Fedora, openSUSE, and Arch Linux AUR. If you run Ubuntu or any of its derivatives such as Linux Mint, ghostwriter can be installed after adding a PPA repository and updating the local cache.

sudo add-apt-repository ppa:wereturtle/ppa
sudo apt update
sudo apt install ghostwriter

If ghostwriter is not pre-packaged for your distro, e.g., Debian, you can follow the on-line instructions to build the executable yourself. Depending on the particular distro and release, be prepared to spend some considerable time as you may run into the proverbial Linux dependency hell.

Feature comparison

Features justmd Remarkable ghostwriter
Cross-platform Linux(x64), Windows(x64), macOS Linux, Windows Linux, Windows
Linux installation Downloadable executables Downloadble packages for Debian, Ubuntu, Fedora, openSUSE, Arch Downloadble packages for Ubuntu, Fedora, openSUSE, Arch
Export to HTML, PDF Yes Yes HTML, PDF, Word, ODT
Spellchecker No No Yes
Auto save No No Yes
Word count No Character, word, line counts Character, word, line, sentence, paragraph, page counts
Live preview Fixed window proportion Hidable, variable proportion Separate resizable window (no dual panel)
GitHub-flavored syntax Support for tables Yes (tables, strikethrough, emphasis, etc) Yes (tables, strikethrough, emphasis, etc)

Summary & conclusion

A writer's working style is intrinsically idiosyncratic. A writing environment that is distraction-free to one person may not be stimulating enough for another. Yet, ghostwriter is the clear winner of the 3 editors because it strikes a balance between clean design and feature richness. However, if ghostwriter is not pre-packaged for your Linux distro (say Debian), justmd and Remarkable are definitely worthwhile alternatives.

Thursday, March 1, 2018

Sharing folders between VirtualBox host and guest machines


This post is the 4th installment of the VirtualBox series. The focus is on how to share folders between the host and the guest OSes. Part 1 of the series shows how to install VirtualBox on a Debian host, and how to create a FreeBSD virtual machine. Part 2 outlines the post installation tasks, including installing the universal VirtualBox extension pack. Part 3 shows how the guest OS can access USB drives mounted on the host.
The method outlined in this blog post works for Linux and Windows guest OSes but not FreeBSD. You can set up NFS or Samba to share a folder between a Linux host and a FreeBSD guest.

Prerequisites

To share a folder using the following procedure, the OS-specific guest additions package must be pre-installed. Part 1 of this series includes instructions for installing the guest additions package for the FreeBSD guest OS.

Procedure

  1. Declare the shared folder on host OS.
    Open the Oracle VM VirtualBox manager on your host machine, select the target guest OS, and click Settings. Note that your guest machine does NOT need to be powered off.
  2. Select Shared Holders, and click the Plus button to add a shared folder. Note that you can have more than 1 shared folder.
  3. Specify the path for the shared folder.
    Click the down arrow next to the Folder Path parameter, select Other, and navigate to the target shared folder.
  4. Configure the shared folder.
    The Folder name field refers to the folder name on the guest OS, which defaults to the folder name you specified in the last step. You should note the folder name in order to identify the full folder pathname on the guest OS. For example, if the shared folder on the host machine is /home/peter/Music, the corresponding folder name defaults to /media/sf_Music on the Ubuntu guest OS, and \\vboxsvr\Music on the Windows guest OS.

    The Auto-mount checkbox should be enabled. Optionally, you can also mount the folder manually, but then you must modify access permissions in order to make the folder writable by non-root users.

    Unless the shared folder is for one-off use only, you should enable the Make Permanent checkbox.
  5. Power recycle the guest machine.
  6. Grant folder access permission to non-root users.
    This step only applies to Linux guest OS(not Windows). Login to the guest OS, and add regular users to the vboxsf group using the following command.

    sudo usermod -aG vboxsf <someuserID>

Related posts

Wednesday, February 14, 2018

Ubuntu: how to reset lost administrative password

After an extended vacation, I came home to discover to my horror that I could no longer login to my seldom-used Ubuntu laptop. The reason was embarrassingly simple: I forgot my password. My muscle memory (or finger memory) did not help while I was frantically typing my usual passwords but to no avail. I own the administrative account on that system. So, I had no one else to turn to for help. Luckily, I was able to login another machine and google how to reset the administrative password on Ubuntu.

Prerequisites

There are 2 preconditions for using the procedure to reset the administrative password.
  1. Physical access to machine.
    You need to access the system console in order to interrupt loading of the OS.
  2. The root password was disabled.
    By default, Ubuntu disables the password of the root account by assigning it a value which cannot possibly match any encrypted value. During the installation of Ubuntu, you were asked to create the first user. That initial user, by default, belongs to the sudo group which means that the user can be elevated to perform system administration functions. The administrative password which we are going to reset refers to the password of the initial user.
    The procedure assumes that you did not manually assign root a valid password. If root has a valid password and you know it, then you can simply sign on as root and reset the administrative password using the password command. If the root password was also forgotten, this procedure does not apply because you need to enter that password to drop to root shell prompt in this procedure.

How to reset administrative password

  1. Power recycle.
    After the BIOS screen appears, press down the left shift key to enter GRUB. This step can be quite finicky, and you may need to repeat it several times until you get the timing just right.
  2. Scroll down to Advanced options for Ubuntu, and press Enter.
  3. In the ensuing screen, scroll down to the top Recovery mode line, and press Enter.
    If the Linux kernel image had been upgraded on the machine before, you would see multiple recover mode lines on the screen. Select the recovery mode line that corresponds to the latest Linux image(that is nearest to the top).
  4. In the Recovery Menu screen, scroll down to root, and press Enter.
  5. Press Enter again.
    If you have previously assigned root a password, you would be prompted to enter it at this step. Otherwise, just press Enter to continue.
  6. Remount filesystem.
    After all the hard work, you are now at the root shell prompt. The filesystem at this point is read-only. Remount the file system to add write permission.
    # mount -o rw,remount /
    
  7. Reset administrative password.
    Use the passwd command to change the password for the administrative user.
    # passwd <adminuser>
    
    Press Control-D to return to the recovery menu.
  8. Select resume.
  9. Exit recovery.
    Press OK to exit recovery mode and continue booting.

Thursday, February 1, 2018

How to mount USB drives on VirtualBox


This is part 3 of the 4-part series on VirtualBox. This post shows you how to access, from your guest OS, a flash drive mounted on the Debian host. Parts 1, 2 and 4 demonstrate how to build a FreeBSD virtual machine using VirtualBox on a Debian host, perform post-installation tasks, and share folders.

Prerequisites

The VirtualBox extension pack must be installed on the VirtualBox host. See part 2 for instructions.

Configuring USB

Follow the steps below to configure VirtualBox USB.
  1. Add user to vboxusers group.
    VirtualBox access to the host's USB drives is only granted to users of the vboxusers group. As root on the host, run the following command to add each VirtualBox user (e.g., peter) to the group.
     # usermod -aG vboxusers peter
    
  2. Power off VM.
    VirtualBox defaults to using USB Controller 1.1 (OHCI). Modern hardware uses USB Controller 2.0(EHCI) and USB Controller 3.0(xHCI). Before you can change the USB controller protocol, the virtual machine must be powered off.
  3. Open VirtualBox Manager, click Settings, and select USB.
  4. Specify USB Controller.
    Select either USB 2.0 (EHCI) Controller or USB 3.0 (xHCI) Controller according to your actual hardware.

    Note that you can add USB Device Filters to define the types of USB drives which will be automatically made visible to the guest OS. Be forewarned that the USB drive, once made visible to the guest OS, will no longer be available to the host. More on the use of device filters in the next section.

Accessing USB drive

Below is the step-by-step procedure to mount and access a flash drive.
  1. Insert the flash drive into your host machine's USB port.
  2. Unmount the flash drive (if it is auto-mounted on your host).
    Making it available to the guest will automatically and instantly unmount it from the host. To avoid any data loss due to pending writes to the drive, it is a good practice to explicitly unmount the drive prior to handing control to the guest.
  3. Power on the FreeBSD guest.
  4. Assign USB drive to guest OS.

    Open the virtual system console, and right click the USB drive icon.

    Click to select your USB drive.

    Note that this is a 1-time assignment only. Please see instructions at the end of the section on how to automatically assign this particular USB drive for all subsequent sessions.
  5. Login to FreeBSD, and mount the drive.
    You can mount a MS-DOS based flash drive by running the following commands as root. Replace /dev/da0s1 with the proper device identifier for your USB drive. (You can find out the exact device ID by first running dmesg to identify the device name, e.g. da0, and fdisk to reveal the disk partition structure, e.g., s1.)
     # mkdir -p  /media/usb
     # mount -t msdosfs  /dev/da0s1  /media/usb
    
    To unmount the drive,
     # umount /media/usb
    
To always automatically assign a particular USB drive to the guest OS, open the VirtualBox Manager, click Settings, and then USB.

Finally, click Add USB device filter (with the + sign) icon, and select the USB drive that is currently inserted in the host.

Related posts