The other day I was toying around with HTTP2 support and Jetty 9.3 on my Ubuntu Server 14.04 VM install. I had mostly followed the helpful instructions in the Jetty: The Definitive Reference on setting up the application as a web server with support for http, https, deploy modules. But whenever I tried to implement HTTP2 support Jetty would fail with an unhelpful Java error dump.
java -jar /opt/jetty/start.jar --add-to-startd=http2
org.elecipse.jetty.start.graph.GraphException: Missing referenced dependency: alpn-impl/alpn-1.8.0_45-internal
org.eclipse.jetty.start.graph.GraphException: Missing referenced dependency: alpn-impl/alpn-1.8.0_45-internal
Usage: java -jar start.jar [options] [properties] [configs]
java -jar start.jar --help # for more information
The Jetty log files and Google were equally unhelpful in finding a solution. But after some painful trial and error I worked out a fix.
The Java error is actually telling you that Jetty module alpn-impl, a requirement for http2is looking for a file dependency that doesn’t exist. The module calling for the dependency can be found at modules/alpn.mod in your jetty home. A look into the file reveals the following.
So alpn.mod is using a constant value based on the Java version to generate the dependency name. As it turns out my install of OpenJDK (openjdk-8-jre-headless) returns an interesting version and this is what is causing the problem.
openjdk version "1.8.0_45-internal"
OpenJDK Runtime Environment (build 1.8.0_45-internal-b14)
OpenJDK 64-Bit Server VM (build 25.45-b02, mixed mode)
When we take a look in modules/alpn-impl/ there is no such file as 1.8.0_45-internal.mod which matches the Java dependency error message we were receiving.
alpn-1.8.0_05.mod alpn-1.8.0_11.mod alpn-1.8.0_20.mod alpn-1.8.0_25.mod alpn-1.8.0_31.mod alpn-1.8.0_40.mod alpn-1.8.0_45.mod alpn-1.8.0.mod
Fortunately the fix is easy.
sudo ln -s alpn-1.8.0_45.mod alpn-1.8.0_45-internal.mod
Now if we try to load the http2 module it works!
java -jar /opt/jetty/start.jar --add-to-startd=http2
ALERT: There are enabled module(s) with licenses.
The following 1 module(s):
+ contains software not provided by the Eclipse Foundation!
+ contains software not covered by the Eclipse Public License!
+ has not been audited for compliance with its license
+ ALPN is a hosted at github under the GPL v2 with ClassPath Exception.
+ ALPN replaces/modifies OpenJDK classes in the java.sun.security.ssl package.
In this guide I will walk through the process of hardening HTTPS connectors used by Apache Tomcat. As unfortunately the default configuration of Ubuntu 14.04 LTS using Tomcat 7 and OpenJDK 7 are vulnerable to a number of attacks and weak encryptions.
You can test your own site’s HTTPS implementation against these weaknesses at Qualys SSL Lab SSL Server Test. With this guide we can hopefully boost a F or even a B grade up to an A grade rating.
This guide is targeted towards Ubuntu 14.04 LTS but should work for other distributions.
Install OpenJDK 8 (Java 8)
I will be using a 3rd party ppa to install OpenJDK. On some setups using 3rd party PPAs could be considered a security risk so use your own discretion. Though this process will not delete existing Java installs so you can always revert back to your original install and configuration. Are PPA’s safe to add to my system…?
The first major attribute change we need to implement is the protocol. In Tomcat 7 and below this is set to use the Java BIO Connector with the value of "HTTP/1.1" or "org.apache.coyote.http11.Http11Protocol".
Tomcat 7 users should change this attribute to use the Java NIO Connector which offers a similar functionality to the BIO (HTTP/1.1) connector with a smaller performance footprint. Tomcat 6 users should keep to the default BIO connector attribute while Tomcat 8 and later already use the NIO connector as default.
The default Java 7 BIO and NIO connectors enable SSLv2 and SSLv3 protocols which are vulnerable to the POODLE attack. Connectors using OpenJDK 8+ have these insecure protocols disabled. All users of Java 7 SHOULD add the following attribute to disable this vulnerability.
sslEnabledProtocols="TLSv1, TLSv1.1, TLSv1.2"
The next attribute we’ll use to harden our setup is to manually select the encryption ciphers the connector is permitted to use when communicating with the a browser in HTTPS. Not all ciphers are equal and so I’ve compiled a list of 40 of the most secure yet compatible ciphers available in Java. This collection uses Forward Secrecy (FS) and disables weak algorithms such as the unauthorised Diffie–Hellman (DH) key exchange.
Unfortunately they break the ability to make successful HTTPS connections for many versions of Internet Explorer on Windows XP as well as on Android 2.3.7 or earlier. But for Java 7 users implementing this list is a must as the default collection used by its BIO and NIO connectors contain some weak and insecure ciphers.
Finally Java 8+ users should add the useServerCipherSuitesOrder attribute and set it to true.
By default Tomcat will use the first acceptable cipher presented by the client browser. But often this selection is not the strongest cipher available or supported. The previous cipher attribute lists the ciphers in a preferential order of strength. So by enabling useServerCipherSuitesOrder Tomcat will probe the client using this ordered sequence until a supported cipher is matched making sure the most secure connection available is always used.
After implementing the changes my harden connector looks like this. Again yours may look different depending on which Java edition is in use.
This entry will guide through the process of creating a self-signed certificate to use on an Apache Tomcat 7 or 8 HTTPS connector. Self-signed certificates allow secure, encrypted HTTPS connections but are not certified by any trusted certificate authority. So first time client connections will receive all kinds of warnings from their web browser. Because of this they are not recommended for use in production environments but are useful for secure LAN traffic or testing HTTPS configurations.
I am using Ubuntu 14.04 LTS and Tomcat 7 but this setup should be similar for other distributions.
Create a keystore and certificates
Firstly create and secure a directory to hold our certificates for Tomcat.
Next we are going to create a 2048-bit RSA key and use it to generate an X509 self-signed certificate. Note the -days argument hard codes a usable duration value into the certificate. A argument of of -days 365 means the certificate is valid for a year.
In the code examples you can replace "example" with a domain or site name of your choosing such as localhost or mydomain.com.
You should receive a warning from your browser but you can ignore these and proceed with the connection.
Congratulations you’re viewing your Tomcat served website over an encrypted HTTPS connection. If the connection over HTTPS didn’t work but you can still connect using standard HTTP. Then you best check your Tomcat logs for any configuration errors.
tail -n 100 catalina.out | less
Configure Tomcat to use port 443 (HTTPS default)
If you want to use HTTPS without the need to append :8443 to the end of the URL then we have to bind port 443 (the default HTTPS port) to Tomcat.
tar is a feature rich but often confusing archiving tool most commonly used on Linux and BSD systems. One of the great benefits of tar over other more modern tools such as 7z, RAR or ZIP is that it is open source, free and platform agnostic. Even more importantly for Linux users tar archives preserve user and file permissions allowing for easy directory backups and restoration.
rsync is a widely used tool for synchronisation so to keep copies of a file on multiple computers the same. Because of its flexibility it has become the defacto standard on Linux and other similar systems. While newer protocols and tools such as Dropbox and BitTorrent Sync overlap with and improve on some of rsync’s capability. Rsync is still relevant today both for syncing and for other function such as local directory duplication and large file copying.
Over the years I have looked around for easy to use MySQL database backup tools that I can quickly and reliably automate. I’ve tried many unsatisfactory solutions but finally I have found something I am happy with, Perscona’s XtraBackup.
Unfortunately for Windows users this is a Linux only solution.
Percona is an enterprise focused database company with the aim of delivering their clients faster and more reliable MySQL solutions. Fortunately for the rest of us they also believe in open source and offer a number of free tools suitable for all MySQL and MariaDB users in addition to their own MySQL server platform.
Below is a Markdown formatted cheat sheet that I wrote which enables Linux users to easily back-up their MySQL compatible databases. With simple instructions on how to create, restore and automate backups. While the instructions were written with Ubuntu/Debian in mind they should be useful for all other distributions with the usual adjustments.
In this entry I will explain my method of improving the Windows Command Prompt and its command-line. To turn it from a crude shell prompt into an ANSI coloured interface with a useful set of shell commands that can commonly be found on Linux and Mac terminals. We will also integrate a number of optional popular programming languages and features. This will require some existing knowledge of Windows and the command-line. But before we get into that let’s go on a trip down memory lane for some context.
(you can skip this if you wish)
In the ancient days of personal computing during the mid to late 1970s and the early days of Microsoft. There was a popular text only operating system called CP/M. By today’s standards it was rather crude and rudimental but at the time it was popular in business due to its relative ease of use.
Years later when Microsoft delivered to IBM its first PC operating system. It became apparent that the Microsoft operating system MS-DOS was a derivative of this earlier CP/M system.
I bring this up because to this day the default and most common Windows command-line the Command Prompt cmd.exe. Can trace its lineage back to this 1970s operating system from the defunct Digital Research.
After Windows became popular Microsoft merged both its MS-DOS shell and Windows overlay together into a operating system known as Windows 95. So for the longest time until the release of Windows 2000 and XP. Most consumers used a version of Windows that had a MS-DOS Command Prompt as a critical component of the operating system.
With Windows 2000, XP and later operating systems the MS-DOS functionality was pushed to the wayside and has been neglected ever since. Since the release of PowerShell in 2006 this neglect deepened. In fact the current iteration of the Command Prompt in Windows 8.1 is not much different to that which was found in Windows 95 nearly twenty years ago.
Windows Command Prompt
So let us start with the default Command Prompt.
In Windows use the Run feature (usually this can be done with the Windows key + R) and type in CMD. This should launch the default Command Prompt application which should be similar to the prompt above.
The Command Prompt CMD.EXE is the application that we use to input and receive feedback from the command-line shell. The shell is a collection of programs that allows you to interact with the Windows filesystem and operating system.
This is an important distinction because we are going to replace the Command Prompt but not the underlying command-line shell.
Replace the Command Prompt CMD.EXE with ConsoleZ
First create a directory on the root of your hard drive to contain all your command-line tools. I will use c:\terminal but some other directory name suggestions could be c:\prompt or c:\cp or c:\shell. I do recommend that you keep the directory name short and use a complete word without any spaces.
Go to the ConsoleZ download page and select the x86 for the 32-bit or amd64 for the 64-bit edition. Save the download to your c:\terminal directory and unzip it.
For all the rest of the article I am going to use and reference the 64-bit edition.
ConsoleZ is a portable application which means it does not need to be installed to work. This means you can easily archive and backup the whole c:\terminal directory into a zip file or copy it to a USB stick or a secondary computer without losing any of your customisations or settings.
With ConsoleZ you have an alternative Command Prompt. I do the following customisations to give it a less cluttered look but you can pick and choose which ever you want.
Menu > View > Toolbar off Menu > View > Status Bar off
Menu > Edit > Settings > Console
Change the Windows size Rows value if you want a longer command-line interface by default.
Change the Buffer size Row value if you want to keep a larger backlog of command-line output. By default 500 lines of text are stored to memory but increasing this value is useful if you want to display and scroll through large logs or text files. I usually increase it to its maximum value of 32766.
Menu > Edit > Settings > Appearance > Styles Window Transparency > Alpha > Active Window
I set the application transparency as a low value of 20 but Windows 8+ users may not wish to use this aesthetic effect.
Customize the ConsoleZ font
Next up is I adjust the font selection which can be an important personal choice depending on your requirements and screen size. The default font is Courier New at a size of 10.
Menu > Edit > Settings > Appearance > Font
I change Name: to use Lucida Console in regular with a Size: of 10. But play around until you find a font and size combination that you like.
Command and Batch Scripts
In c:\terminal create a new subdirectory named cmd-scripts ie c:\terminal\cmd-scripts. This will contain my custom text based scripts that use the file extension .bat or .cmd. Modern Windows scripts use the .cmd (Command) extension while the .bat (Batch) is a MS-DOS legacy convention that functions exactly the same.
Colourise Your Command-line with ANSI Colour
In the 1980s and 1990s Windows and MS-DOS supported a widely used feature called ANSI escape sequences that allowed additional functionality such as colour text. But this support was dropped in more recent Windows editions. So I will show you how to re-implement ANSI escape sequence support using Jason Hood’s excellent ANSICon shell overlay.
Now if you run the command type "ANSI Prompt Colours.txt" you should see a whole lot of garbled text dumped to the ConsoleZ window. These are ANSI escape sequences combined with plain text that Windows does not know how to interpret.
Download then save ANSICon to your c:\terminal directory and unzip it into its own ansicon directory. I keep it in c:\terminal\ansicon.
If you are using 32-bit Windows or 32-bit ConsoleZ you should replace the following code c:\terminal\ansiconx64\ansicon.exe with c:\terminal\ansiconx86\ansicon.exe.
Back in ConsoleZ we will set the shell-colour.cmd as our default shell. This is a hack that loads ANSICon in addition to the default cmd.exe shell to give us ANSI escape sequence support without replacing the underlying shell.
Menu > Edit > Settings > Console
Under Shell: add c:\terminal\cmd-scripts\shell-colour.cmd
Now load a new tab.
Menu > File > New Tab > Console 2
Or relaunch ConsoleZ. In c:\terminal display the ANSI Prompt Colours.txt using the following commands.
type "ANSI Prompt Colours.txt"
If you see coloured text congratulations you now have ANSI escape sequence support.
Customise and Colourise the Prompt
The default Windows text input prompt only lists the active drive and current directory. As a frequent Linux Bash shell user I like a bit more flare and information to my prompt.
This code adds a couple of new commands. The echo command displays text to your command-line while the prompt command customises the input text prompt. The strings that are wrap within percentage symbols % are environment variables that are accessible from the shell. They allow you to display tidbits of information that are stored by Windows to the computer memory. A complete list of environment variables can be found at SS64.com.
The prompt command has some rather cryptic ANSI escape sequences that introduce colours. The $E string is a prompt argument to display escape characters which conveniently is needed by the ANSI escape sequences as a trigger. The [number;number;40m is a code sequence used to trigger an effect. A list of ANSI escape effects and colours is listed on Pueblo.
Reload ConsoleZ or open up a new tab to apply the changes. You should see a more information pack and colourful input text prompt.
The first part of the prompt in green displays the USER @ DOMAIN while the second part in blue displays the active drive and path.
Set a Default Directory at Launch
By default ConsoleZ sets the active directory to the location of its application which in my case is c:\terminal\ConsoleZ.x64. But you can easy change this by doing the following.
Menu > Edit > Settings > Console
Set Startup dir: to the path of your choosing such as c: or you can use a Windows environmental variable such as %userprofile%.
Create a Run as Administrator tab
A neat feature of ConsoleZ is the ability to create shell tabs with different user account permissions including those of an Administrator. As there are times when less restricted access is required to interact with parts of the Windows and its settings.
In ConsoleZ do.
Menu > Edit > Settings > Tabs > Add
In the Main tab change the Title: value to something more meaningful such as Administrator Console.
Point the Icon: to C:\Windows\System32\imageres.dll and select the yellow and blue shield icon which represents Run as administrator.
Under Shell make sure the Run as current user option is selected and the Administrator checkbox is checked.
Press OK when done. You now have a new Run as administrator tab accessible from ConsoleZ.
To run most tools and software from the command-line the active directory has to be the same as the location of the tool.
For example if I am in C:and try to run our script shell-colour.cmd that is located in c:\terminal\cmd-scripts. The shell returns the error “’shell-colour.cmd’ is not recognized as an internal or external command, operable program or batch file”. Which basically means the shell could not find the command I was trying to run.
But by using the Windows environment variable called %PATH% we can add additional directories that the shell will scan in when it searches for the existence of programs.
The pane on the left contains a list of editable Windows system variables. While the pane to the right has variables that are restricted to your current Windows user account. You may notice there is a PATH= variable for both the system and the user panes.
I generally use the system PATH for my c:\terminal path links. But to modify this you need to run Rapid Editor in administrator mode.
By default Windows stores multiple values of the PATH in a single string separated by semicolons ;. Fortunately Rapid Editor allows you to list and edit each individual PATH entry.
To add new directories to the PATH, right-click the PATH= string in Rapid Editor and select Add value or use the Alt+Ins keyboard combination.
Press F7 or select Insert directory path… Then point it to your c:\terminal\cmd-script directory. You should now have a new entry in your PATH list. Save the changes by pressing the Save icon or using Ctrl+S.
Now for changes to have an effect you have to either restart ConsoleZ or open up a new tab. To test that the PATH modifications are active, issue a cd command to return to the directory root and try running the shell-colour command again. If it works then congratulations you can now run any custom scripts placed into c:\terminal\cmd-script from anywhere in your command-line.
Launch Windows Notepad++ or Notepad from the Command-line
While I love using command-line shells I generally prefer a GUI when it comes to programming or editing text files. I use the following script to to edit a file from the command-line. It launches Notepad++ and opens the file supplied but you can use any text editor including Windows Notepad located in C:\Windows\System32\notepad.exe.
In a new session of ConsoleZ test the edit script with the following commands.
edit "ANSI Prompt Colours"
To regain your prompt either close Notepad++ or press Ctrl+c in ConsoleZ.
Congratulations the core of this guide is complete. The remainder paragraphs are optional so you can pick and mix which topics you wish to implement. These include adding extra Windows and Linux tools; installing and running Node.js, Perl, PHP, Python or Ruby scripts from the command-line.
Useful Windows Command-line Tools
Besides the complete collection of commands and command-line programs built into Windows that you can discover at http://technet.microsoft.com/en-us/library/bb490890.aspx and http://ss64.com/nt/. There are quite a number of useful third-party command-line programs that are available for the Windows platform. I like to keep these contained in a single directory at c:\terminal\bin. Bin is an abbreviation for binary an alternative term for a program file.
Create the directory c:\terminal\bin and then using the processes covered in ‘Configuring Paths’ add c:\terminal\bin to your PATH variable. Restart ConsoleZ and so that any programs placed into c:\terminal\bin should be accessible from anywhere within your shell.
Here are a few Windows native command-line tools I recommend.
Linux/Unix Terminal Command-line Tools for Windows
As a frequent Linux Bash shell user there are a number of tools I miss on Windows. Fortunately as much of Linux is open sourced many of those tools have been ported over. Unfortunately some of these ports are horribly out of date so they should be best avoided.
There are also a couple of popular Windows open source C compilers such as Cygwin and MinGW that contain ports of Linux terminal tools. But in my opinion their use are overkill and many of the ported tools are rather old.
Finally any useful terminal needs a copy of OpenSSH for remote logins. While there are many out of date variants of OpenSSH on Windows. I personally use OpenSSH for Windows ported by mls-software.com which is the most up to date port I have found. Simply download the current ‘New’ version from their website, the download link is named setupssh-6.[version].exe and run the setup installer. The installer should automatically configure the %PATH% for you.
To test OpenSSH just output its version. If it doesn’t match the version number listed on mls-software.com site (6.7 p1 as of writing) then you probably have one or more copies of OpenSSH on your system that are taking president in your %PATH%. Tools such as Cygwin or Git can have their own out of date ports of OpenSSH.
The popular Linux synchronisation tool rsync also has a Windows port.
You can refine this one step further so you don’t even have to type the .php file extension. Open the Rapid Editor and under System variables add the value .PHP to the PATHEXT variable. In a new ConsoleZ tab you should be able to run hi-php without the file extension.
You can refine this one step further so you don’t even have to type the .pl file extension. Open the Rapid Environment Editor and under System variables add the value .PL to the PATHEXT variable. In a new ConsoleZ tab you should be able to run hi-perl without including the file extension.
Run Python Scripts and Programs from the Windows Command-line
The default Windows download of Python 3.4+ automatically configures itself to enable you to run Python scripts and programs from the Windows command-line.
Run Ruby Scripts and Programs from the Windows Command-line
At the time of writing Ruby is a little bit behind on the Windows platform. The current build of Ruby is at edition 2.1.x, while the recommended edition to use on Windows is 1.9.x. So with this in mind I recommend using the RubyInstaller for Windows which is a self-contained Ruby install.
Run the setup program and when prompted make sure you select both checkboxes for the options below.
Add Ruby executables to your PATH Associate .rb and .rbw files with this Ruby installation
Another great feature of ConsoleZ is because it offers separate, tabbed environments you can use it as an interactive programming tool.
In ConsoleZ do the following.
Menu > Edit > Settings > Tabs > Add
In the Main tab change the Title: value to Python 3
Point the Shell: to the language interpreter. For a default Python 3.4 installation I have it pointed to c:\python3\python.exe.
Under Main you can set the Icon: value which usually should be the same as the Shell: value.
When done, press OK and you now have a new Interactive Python shell tab accessible from ConsoleZ.
For an interactive Ruby shell you need to set the Shell: value to c:\ruby193\bin\irb.bat (or wherever your Ruby installation is located). And you probably want to set the Icon to c:\ruby193\bin\ruby.exe.
PowerShell in ConsoleZ
While this guide mostly uses the standard CMD.EXE shell. Microsoft provides an alternative, feature rich and overall better shell known as PowerShell. PowerShell is an optional download for most consumer versions of Windows and offers a more powerful and complicated command-line. Fortunately PowerShell can also easily be incorporated into ConsoleZ.
Within ConsoleZ do the following.
Menu > Edit > Settings > Tabs > Add
Add a new tab and set the Title: to PowerShell.
Set the Shell: value to %SystemRoot%\system32\WindowsPowerShell\v1.0\powershell.exe
Set the Icon: to use that same file path.
If you want you can also create an administration version of this tab. Select your newly created PowerShell tab and click the Clone button. Change the Title: to Administrator PowerShell and the Icon: to use the PowerShell icon with Administrator shield. Finally make sure the Run as current user option also has the Administrator check-box checked.