TLS and SSL certificates cheat sheet

This is a summary for getting a properly-signed certificate to use with a web server. Probably the same or similar process for other servers, e.g. LDAP.

1. Generate an RSA private key for your server.
2. Create a Certficate Signing Request (CSR), specifying the above RSA private key as the key.
1. Verify the CSR.
3. Send the CSR to a Certificate Authority (CA).
4. The CA will send back:
1. Your signed SSL certificate, in several files in several formats. Our CA gives these four:
1. Certificate with chain, PEM encoded - foobar_example_com.cer
2. Certificate only, PEM encoded - foobar_example_com_cert.cer
3. Certificate as PKCS#7, PEM encoded - foobar_example_com.crt
4. Certificate as PKCS#7 - foobar_example_com.p7b
2. An intermediate CA certificate file. This can be thought of as a child of the root CA certificate, which is private and protected by the CA. Our CA gives this file:
1. Root/Intermediate(s) only, PEM encoded - foobar_example_com_interm.cer
3. Possibly a reverse intermediate CA certificate. I am actually not certain what this is.

In the following, let’s say that the FQDN of your server is foobar.example.com.

On a Red Hat-like system, all the SSL- and TLS-related files/certificates are in the directory tree based at:

/etc/pki/tls/

To generate the RSA private key file foobar.example.com.key:

# cd /etc/pki/tls/private

# openssl genpkey -algorithm RSA -out foobar.example.com.key -pkeyopt rsa_keygen_bits:2048

The private key is now in:

/etc/pki/tls/private/foobar.example.com.key

Keep this key private, i.e. root-only access.

Next, create a CSR using that newly-created key, also specifying the FQDN to be associated with the certificate that you are requesting:

# cd /etc/pki/tls/certs

# openssl req -sha512 -new -key /etc/pki/tls/private/foobar.example.com.key -out foobar.example.com.csr

The CSR is the file foobar.example.com.csr.

Verify the CSR:

# openssl req -noout -text -in foobar.example.com.csr

You will get output showing information in the request. Check for the “Subject” line, that it matches your geographical and company information, etc.

Send the CSR to your CA of choice. They will use that CSR to sign a certificate. The signed certificate will be sent to you, along with an intermediate CA certificate, and possibly a reverse intermediate CA certificate. 

The CA we use at my organization sends back three files:

• foobar_example_com_cert.cer - the signed certificate
• foobar_example_com_interm.cer - the intermediate CA certificate
• foobar_example_com_interm_reverse.cer - the reverse intermediate CA certificate

For Red Hat-like systems, these files should be put in:

• signed certificate - /etc/pki/tls/certs/foobar_example_com_cert.cer
• intermediate CA certificate - /etc/pki/tls/certs/foobar_example_com_interm.cer
• reverse intermediate CA certificate - /etc/pki/tls/certs/foobar_example_com_interm_reverse.cer

For Apache HTTPD setup, modify the file /etc/httpd/conf.d/ssl.conf

<VirtualHost _default_:443>

…

SSLCertificateFile /etc/pki/tls/certs/foobar_example_com_cert.cer

SSLCertificateChainFile /etc/pki/tls/certs/foobar_example_com_interm.cer

…

</VirtualHost>

There is one more setting in that VirtualHost section, for the CA certificate bundle: the SSLCACertificateFile. This file is usually provided by a distro package. In the case of RHEL, it is provided by the ca-certificates package. The default value should be:

SSLCACertificateFile /etc/pki/tls/certs/ca-bundle.crt

Sources:

• https://www.thesslstore.com/blog/root-certificates-intermediate/
• http://www.sobit.org/ssl_cs.pdf 
• About the file suffixes CRT, CER, PEM, DER: https://support.ssl.com/Knowledgebase/Article/View/19/0/der-vs-crt-vs-cer-vs-pem-certificates-and-how-to-convert-them 

Neocortex at Pittsburgh Supercomputing Center - 40k-core custom CPU

Pittsburgh Supercomputing Center (PSC) has a new custom-designed 40k-core AI/big data system called Neocortex. Pretty scarily impressive. Here is a presentation on it: 

Scripting Bright Cluster Manager 9.0 with Python

It has been more than 6 years since the previous post about using the Python API to script Bright Cluster Manager (CM). Time for an update.

I have to do the same as before: change the “category” on a whole bunch of nodes.

NB the Developer Manual has some typos, where it makes it look like you can specify categories as strings of their names, e.g. cluster.get_by_type('Node')

Building Ganglia Monitor Core 3.6.1 on RHEL 7 - "Installed (but unpackaged) file(s) found" error

Install a bunch of prerequisites, some from EPEL:

* libconfuse

* libart_lgpl-devel

Download source: monitor-core-3.6.1.tar.gz

Expand the tarball: tar xf monitor-core-3.6.1.tar.gz  That creates the directory monitor-core-3.6.1

Enter that directory and run the bootstrap script: ./bootstrap.sh

That generates the configure script. 

Run the configure script: ./configure

That generates the SPEC file: ganglia.spec

Make a tar ball with the appropriate name:

    cd ..

    mv monitor-core-3.6.1 ganglia-3.6.1

    tar zcf ganglia-3.6.1.tar.gz ganglia-3.6.1

Build with rpmbuild -ta ganglia-3.6.1.tar.gz

Will probably get RPM build errors:

    bogus date in %changelog: Thu Mar 28 2008 Brad Nicholes <bnicholes@novell.com>

    bogus date in %changelog: Wed Jul 10 2007 Bernard Li <bernard@vanhpc.org>

    bogus date in %changelog: Wed Jul 3 2007 Brad Nicholes <bnicholes@novell.com>

    bogus date in %changelog: Wed Jun 14 2007 Brad Nicholes <bnicholes@novell.com>

    bogus date in %changelog: Fri Feb 25 2006 Bernard Li <bli@bcgsc.ca>

    Installed (but unpackaged) file(s) found:

   /usr/lib/systemd/system/gmetad.service

   /usr/lib/systemd/system/gmond.service

Modify the SPEC file ganglia.spec: add line in "%files gmetad" section:

    /usr/lib/systemd/system/gmetad.service

and in "%files gmond" section:

    /usr/lib/systemd/system/gmond.service

Make a new tar ball containing the fixed SPEC file (delete the old one first):

    rm ganglia-3.6.1.tar.gz

    tar zcf ganglia-3.6.1.tar.gz ganglia-3.6.1

Then build with: rpmbuild -ta ganglia-3.6.1.tar.gz

RPMs should be in: ~/rpmbuild/RPMS/x86_64

Dealing with Excel CSV/TSV files in UTF-16LE encoding, and an invisible character

Motivation: I am trying to read a TSV file produced by Blackboard courseware. I am trying to do so using Python's built-in csv library. I am using a simple method:

import csv 

with open('myfile.tsv', 'r', encoding='utf-16le') as cf:
    cr = csv.DictReader(cf, dialect='excel-tab')
    print(cr.fieldnames)   

    for row in cr:       
        print(row)

What I got for the fieldnames was:

['\ufeff"Last Name"', 'First Name']

I.e. the "Last Name" field got munged with \ufeff or U+FEFF in normal Unicode notation.

and the rows looked like:

OrderedDict([('\ufeff"Last Name"', 'Doe'), ('First Name', 'Alice')])

Just using vim to look at the file, there seemed to be nothing weird about the first line which contains the column names. This is a ZERO WIDTH NO-BREAK SPACE character used as a Byte Order Mark (BOM). It allows reading processes, e.g. file(1)/magic(5), to figure out the byte order of the file.

But it messes up the csv.DictReader parsing of the field names.

Turns out, there is an easy fix. You can just modify the field names directly:

cr = csv.DictReader(cf, dialect='excel-tab')
cr.fieldnames[0] = 'Last Name'

And the output is fixed, too:

OrderedDict([('Last Name', 'Doe'), ('First Name', 'Alice')])

RHEL7 on VirtualBox graphics controller

I have a RHEL7 VM on VirtualBox 6.0 which crashed right after a full update to RHEL 7.7, including kernel. Just typed "startx" (I usually make VMs boot into console), and it crashed immediately with a "Guru Meditation" from VirtualBox.

Taking a stab, I changed the Graphics Controller for the VM from VMSVGA to VBoxVGA, and it launched the GUI just fine.

Even more about SSSD + PAM + LDAP -- password is still expired even right after being changed by user

This keeps coming back to haunt me, partly because of patchy and disparate documentation, and partly because I do not have a rock-solid understanding of all the details of SSSD + PAM + LDAP. (Previous post.)

This is for RHEL6.

Here is the issue: my users kept running into the instance when upon logging in, they were shown:

WARNING: Your password has expired.
You must change your password now and login again!
Changing password for foouser.
Current password:

And then it automatically logs you out, which is expected behavior.

However, when they login again (with the password that they just set), they are again presented with the same password expiration warning. This repeats ad infinitum.

When I check the OpenLDAP server, and ldapsearch for the user record, it does show that the password was changed by that user on the correct date.

The key bit that I seem to have missed: a setting in /etc/pam_ldap.conf You have to set the secure LDAP URI since SSSD password transmissions must be encrypted.

uri ldaps://10.9.8.7/

This should match the URI specified in /etc/openldap/ldap.conf

URI ldaps://10.9.8.7/

And the setting in /etc/sssd/sssd.conf

[domain/default]
...
ldap_uri = ldaps://10.9.8.7/
...

And that fixed it.

While you are at it, you might as well specify SHA512 for the hash in /etc/pam_ldap.conf

pam_password sha512

I RTFMed: "sha512" is not an option for pam_password. This is to hash the password locally, before passing on to the LDAP server. The default is "clear", i.e. transmit the password in the clear to the LDAP server, and assume the LDAP server will hash if necessary. Another option is "crypt" which uses crypt(3).

pam_password crypt

However, there does not seem to be a way to specify which hash algorithm is to be used.

I do not think this is a big issue because the connection to the LDAP server is encrypted, any way.

Why was this a surprise? Well, because in /etc/nsswitch.conf we specified sss as the source for the passwd, shadow, and group name services:

passwd:     files sss
shadow:     files sss
group:      files sss

I.e., everything should be mediated through SSSD, and the SSSD config does have the correct URI.