2016-11-08

scikit-learn with shared CBLAS and BLAS

If you have your own copies of BLAS and CBLAS installed as shared libraries, the default build of scikit-learn may end up not finding libblas.so which libcblas.so depends on.

You may, when doing "from sklearn import svm",  get an error like:

from . import libsvm, liblinearImportError: /usr/local/blas/lib64/libcblas.so: undefined symbol: cgemv_

To fix it, modify the private _build_utils module:

scikit-learn-0.xx/sklearn/_build_utils/__init__.py

--- __init__.py.orig    2016-11-08 16:19:49.920389034 -0500
+++ __init__.py 2016-11-08 15:58:42.456085829 -0500
@@ -27,7 +27,7 @@

     blas_info = get_info('blas_opt', 0)
     if (not blas_info) or atlas_not_found(blas_info):
-        cblas_libs = ['cblas']
+        cblas_libs = ['cblas', 'blas']
         blas_info.pop('libraries', None)
     else:
         cblas_libs = blas_info.pop('libraries', [])

2016-11-07

Optimized zlib

I wasn't aware of optimized versions of zlib, the free patent-unencumbered compression library, until today. I ran across Juho Snellman's comparison benchmarks of vanilla zlib, CloudFlare zlib, Intel zlib, and zlib-ng. The upshot is that CloudFlare's optimizations seem to be the best performing. And its decompression times were 75% of the vanilla version, while Intel and zlib-ng ran at 98% and 99% respectively. This would be a clear win for read-intensive workflows, such as some bioinformatics workflows.

Read more about how CloudFlare was contacted by the Institute of Cancer Research in London to help improve zlib at this blog post by Vlad Krasnov. Intel has an overview of zlib in its Intel Performance Primitives (IPP) product. And this is zlib-ng's GitHub repo.

2015-11-12

docbook2x-texi

Building Git 2.6.3 from source, I ran into this error:
    DB2TEXI user-manual.texi
/bin/sh: line 1: docbook2x-texi: command not found
This is the docbook to texi converter from the docbook2X package (which has not been updated since 2007).

For Red Hat and Fedora, the EPEL package repo provides docbook2X. However, the name of the script is changed because there is a newer docbook package. So, in the git source directory, edit the file Documentation/Makefile and change one line:

DOCBOOK2X_TEXI = db2x_docbook2texi

2015-11-11

SWIG is great - Python DRMAA2 interface in less than an hour

I have never used SWIG before, surprisingly. I figured creating a Python interface to DRMAA2 would be a good self-tutorial. Turns out to be almost trivial, following the directions here.

My Python (3.5) DRMAA2 interface code is on GitHub - https://github.com/prehensilecode/pydrmaa2. The hardest part, really, was writing the Makefile.

NOTE: no testing at all has been done. This is just a Q&D exercise to use SWIG.

2015-10-13

Autotools mythbuster

This is a really useful site by Diego Elio “Flameeyes” Pettenò that clarifies and provides good examples of autotools usage: https://autotools.io/index.html

2015-08-21

Modernized makefiles for VASP 5.3

One of the research groups on the cluster I manage requested help in compiling VASP 5.3 (the Vienna Ab-intitio Simulation Package), code for electronic structure calculations and QM molecular dynamics from first principles. Like many research codes, the build procedure is a bit difficult owing to legacy stuff.

I updated the makefiles for my build, to more current GNU Make standards. They worked for me; however, I make no guarantee that they will work for you. In fact, they will likely not work for you without modification. In any case, the repo is on GitHub. Fork away!

    https://github.com/prehensilecode/vasp_makefiles_proteus