Notes on Ubuntu (Linux) computing

Brief notes on installation, setup and other tasks in the Ubuntu world

Archive for the ‘computing’ Category

Install CherryPy on Ubuntu 12.04

leave a comment »

CherryPy “is a pythonic, object-oriented web framework”. My goal is to use the package to make a simple desktop, browser-based application for looking at the output of statistical inference routines I’ve developed elsewhere — we’ll see if this works (I’m not sure yet).

In any case, the first step is to install with pip:

sudo pip install CherryPy

This should install without issue. There are tutorial scripts located at /usr/local/lib/python2.7/dist-packages/cherrypy/tutorial/ that you should be able to run. For example, the `hello world’ example can be started typing the following at you terminal:

python /usr/local/lib/python2.7/dist-packages/cherrypy/tutorial/tut01_helloworld.py

This will start up the web service and the resulting output can be seen in your browser by going to http://localhost:8080. To exit, close the web browser and use a Cntrl-C at the terminal to stop the server.

As I said above, I’d like to be able to read the contents of local files and display them in a browser. If a local file is called numbers.dat and contains the following:

col1,col2
1,5
2,6
3,2
4,7

We can use the following script to display the contents of the above file:

#! /usr/bin/env python
# -*- coding: utf-8 -*-
# vim:fenc=utf-8
#

"""A super simple example of reading a file and displaying results with
CherryPy.

"""
import cherrypy

class ReadFile(object):
    def index(self):
        page = "<html><body>\n"
        page += "<p>Read file: <i>numbers.dat</i></p>\n"

        with open("numbers.dat", "r") as f:
            for line in f.readlines():
                try:
                    col1, col2 = line.strip().split(',')
                    page += "<p>{:s} -- {:s}</p>\n".format(col1, col2)
                except:
                    pass

        page += "</body></html>"

        return page

    index.exposed = True

# start CherryPy
cherrypy.quickstart(ReadFile())

For more detailed documentation and tutorials, see the CherryPy website.

Advertisements

Written by Chris S

January 23, 2014 at 9:49 pm

Install memory_profiler from github repository

leave a comment »

In order to do memory profiling of Python scripts I am installing memory_profile. It is recommended that psutil be installed– we covered this in a previous post.

To install memory_profiler we first clone the github repository (as always, do this is a sensible directory):

git clone git@github.com:fabianp/memory_profiler.git

Next, as usual, cd into the new directory and install:

cd memory_profiler
sudo python setup.py install

Finall, following the README file at the repository, we create the file example.py with the following contents:

"""
An example for memory_profiler. Use with the following command:

    python -m memory_profiler example.py

"""
@profile
def my_func():
    a = [1] * (10 ** 6)
    b = [2] * (2 * 10 ** 7)
    del b
    return a
                 
if __name__ == '__main__':
    my_func()

The decorator @profile allows a memory profile for my_func() using the following command from the terminal:

python -m memory_profiler example.py

This will result in the following output:

Filename: example.py

Line #    Mem usage    Increment   Line Contents
================================================
     7    9.621 MiB    0.000 MiB   @profile
     8                             def my_func():
     9   17.258 MiB    7.637 MiB       a = [1] * (10 ** 6)
    10  169.848 MiB  152.590 MiB       b = [2] * (2 * 10 ** 7)
    11   17.262 MiB -152.586 MiB       del b
    12   17.262 MiB    0.000 MiB       return a

A script, mprof, is also installed. This can be used on the same example.py file created above using the following commands:

mprof run --python example.py
mprof plot

This will use matplotlib to create a memory versus time plot of memory usage:

Plot produced by mprof.

Plot produced by the mprof commands above.

See the repository website for more information and examples.

Written by Chris S

December 30, 2013 at 11:16 pm

Posted in Python

Tagged with , ,

Install psutil from google repository

with 3 comments

In order to do some profiling of Python scripts (memory profile, in particular) I am installing psutil and memory_profiler. A nice overview of profiling, both time and memory, can be found here.

To start, we install psutil. First, clone the repository using mercurial (make sure to execute the following in sensible place):

hg clone https://code.google.com/p/psutil/

Next, cd into the new directory and install:

cd psutil
sudo python setup.py install

Finally, start Python and try the following commands:

import psutil
print psutil.cpu_times()
print psutil.virtual_memory()

These commands should provide information you would normally obtain using the top command. Check the website for further examples.

Written by Chris S

December 30, 2013 at 9:52 pm

Posted in Python

Tagged with , ,

Install vim-template vim plugin

leave a comment »

The vim-template plugin allows for the use of templates for different file types: *.html, *.py and so on. Following the installation instructions (see above link), we use pathogen to install the plugin:

cd ~/.vim/bundle
git clone git://github.com/aperezdc/vim-template.git

Use of plugin

Now, you can make use of the templates when starting vim. For example, a python template is loaded by recognition of the *.py filename:

vim test.py

Or, if you have new buffer, type the following for the template to be loaded inside vim:

:Template py

The available templates can be seen here. Using the pathogen installation described above, these templates are located (on your machine) at ~/.vim/bundle/vim-template/templates/.

Customization

If you want to customize certain fields, like email and username, you can add the following to your .vimrc file:

" Customize the settings for vim-template plugin                                
let g:email = "desiredemail@gmail.com"
let g:user = "Desired Name"                                         
let g:license = "Desired License"

If you want to customize the templates, there is a search order for templates (see here) allowing you to write your own template-files and store them in the local directory.

Written by Chris S

November 7, 2013 at 7:28 pm

Posted in computing, Java, LaTeX, Python, R

Tagged with , , , ,

Install development version of scikit-learn

leave a comment »

In this post I will document my installation of (the bleeding edge, development version) of scikit-learn. You might not want to do this — there are stable releases with installation instructions available at the package website. In fact, there are Ubuntu/Debian-specific installation instructions. I would suggest using the Ubuntu package unless you know that you need that latest code. The reason for this is that dependencies will be taken care of by the package manager; otherwise you will have to figure out the dependencies yourself.

Now on to the specifics, assuming you want to install this version. First, change to a directory where you want the git repository to live, and clone the github repository:

git clone git://github.com/scikit-learn/scikit-learn.git

Following the instructions, change directory to the repository and use the included Makefile

cd scikit-learn
make

This will build the package locally and run a bunch of tests (~2600 using nosetests) to make sure that everything works on you computer.

The commands in the Makefile do not install the package on the system. Instead, the scikit-learn website suggests including the path of the (built) package in your PYTHONPATH. I will do this in a slightly unconventional way that does not require admin privileges, allows the path to be included for a specific version of python, for a single user, and employs basic bash commands (this follows ideas from here — look for subsection on The bash way).

First, find out where your version of python has the user site-packages directory.

python -m site --user-site

If you have multiple versions of python installed and just want to affect python 2.6 you would use python2.6 -m site --user-site instead. Next, the directory provided by the above command may not exist. To make sure that it does, make the needed directory:

mkdir -p $(python -m site --user-site)

Again, substitute python with python2.6 in the above if you need to be more specific. Finally, we make a mypath.pth file in this directory that allows python to find scikit-learn:

echo "/home/username/gitLocal/scikit-learn" >> $(python -m site --user-site)/mypath.pth

This assumes that you cloned the scikit-learn repository in ~/gitLocal/. If you did not, change the above to reflect the location for you computer. Again, change the version of python employed, it needed.

Everything should be setup now. To test, cd to you home directory (make sure you are not in the scikit-learn directory) and start python:

cd ~
python

Now, import scikit-learn (hopefully without error):

import sklearn

At this point you can dig into the documentation and examples available at the website.

Written by Chris S

November 6, 2013 at 8:24 pm

Garmin forerunner and Ubuntu 12.04 — updated

leave a comment »

This post is an update to a previous post on getting information from my Garmin forerunner 305 using tools in Ubuntu 12.04. A comment on the previous post pointed out that svn repository I used was no longer working. This is not my repository, so I can’t do anything about that. However, I can make a github repository (the code seems to allow for free distribution) with the code I obtained in 2012. In the interest of being thorough I will provide complete instructions here, with updates as needed.

1 To download running information from my Garmin forerunner 305, I will use garmin-forerunner-tools:

sudo apt-get install garmin-forerunner-tools

2 To check the xml schema, we need xmllint. To make this utility available install libxml2-utils:

sudo apt-get install libxml2-utils

3 To do the file conversion between *.gmn and *.tcx we need utilities developed by others. I have made a github repository for them and you can obtain them using:

git clone git@github.com:cstrelioff/garmin-dev.git

If you don’t have git installed, use:

sudo apt-get install git

4 Finally, we have to make use of the utilities we have installed. One way to do this is to make a directory to save data, for example:

mkdir ~/GarminData
cd ~/GarminData

To extract data, connect the watch via USB and run:

garmin_save_runs

At this point you should see that there are *.gmn files in the GarminData directory that use the directory structure: Year/Month/filename.gmn .

Finally, we need to create *.tcx file that can be uploaded to Garmin Connect. To do this, we need to go to the directory where we placed the git repository (you should be able to execute ls in the directory and see the script gmn2tcx). The basic format of a command is:

./gmn2tcx ~/GarminData/Year/Month/filename.gmn > runfilname.tcx

A couple of important things to note, using the command as above, are:

  • the full path name must be provided for the input file
  • the output file runfilename.tcx will be placed in the same directory as gmn2tcx

Written by Chris S

October 21, 2013 at 6:17 pm

Install mincemeatpy from github repo

leave a comment »

mincemeatpy implements the MapReduce algorithm in Python. Installation from the repository on github goes as usual.

Clone the repository in a convenient location:

git clone git://github.com/michaelfairley/mincemeatpy.git

Change to source directory and install:

cd mincemeatpy/
sudo python setup.py install

Hopefully everything installs smoothly. This means you should be able start python from the command line and import mincemeat without error.

For ideas on how to use this code, consider the following examples elsewhere:

Also of related interest is the mincepie package that allows for use of mincemeatpy on slurm clusters (I have not tried this out yet):

Written by Chris S

October 18, 2013 at 7:46 pm

Posted in computing, Python

Tagged with ,