Notes on Ubuntu (Linux) computing

Brief notes on installation, setup and other tasks in the Ubuntu world

Posts Tagged ‘profile

Install memory_profiler from github repository

leave a comment »

In order to do memory profiling of Python scripts I am installing memory_profile. It is recommended that psutil be installed– we covered this in a previous post.

To install memory_profiler we first clone the github repository (as always, do this is a sensible directory):

git clone git@github.com:fabianp/memory_profiler.git

Next, as usual, cd into the new directory and install:

cd memory_profiler
sudo python setup.py install

Finall, following the README file at the repository, we create the file example.py with the following contents:

"""
An example for memory_profiler. Use with the following command:

    python -m memory_profiler example.py

"""
@profile
def my_func():
    a = [1] * (10 ** 6)
    b = [2] * (2 * 10 ** 7)
    del b
    return a
                 
if __name__ == '__main__':
    my_func()

The decorator @profile allows a memory profile for my_func() using the following command from the terminal:

python -m memory_profiler example.py

This will result in the following output:

Filename: example.py

Line #    Mem usage    Increment   Line Contents
================================================
     7    9.621 MiB    0.000 MiB   @profile
     8                             def my_func():
     9   17.258 MiB    7.637 MiB       a = [1] * (10 ** 6)
    10  169.848 MiB  152.590 MiB       b = [2] * (2 * 10 ** 7)
    11   17.262 MiB -152.586 MiB       del b
    12   17.262 MiB    0.000 MiB       return a

A script, mprof, is also installed. This can be used on the same example.py file created above using the following commands:

mprof run --python example.py
mprof plot

This will use matplotlib to create a memory versus time plot of memory usage:

Plot produced by mprof.

Plot produced by the mprof commands above.

See the repository website for more information and examples.

Written by Chris S

December 30, 2013 at 11:16 pm

Posted in Python

Tagged with , ,

Install psutil from google repository

with 3 comments

In order to do some profiling of Python scripts (memory profile, in particular) I am installing psutil and memory_profiler. A nice overview of profiling, both time and memory, can be found here.

To start, we install psutil. First, clone the repository using mercurial (make sure to execute the following in sensible place):

hg clone https://code.google.com/p/psutil/

Next, cd into the new directory and install:

cd psutil
sudo python setup.py install

Finally, start Python and try the following commands:

import psutil
print psutil.cpu_times()
print psutil.virtual_memory()

These commands should provide information you would normally obtain using the top command. Check the website for further examples.

Written by Chris S

December 30, 2013 at 9:52 pm

Posted in Python

Tagged with , ,