Languages  >>  Python

Tutorial: Scientific and parallel computing using IPython

Language: English Quality: High Has Audio: true Source: Other Media: Flash Posted On: 22 Jun 09
This series introduces scientific and parallel computing using IPython with emphasis on IPython on a Windows PC. We discuss best practices for effectively using IPython with numpy, scipy, and matplotlib, as well has using IPython for interactive parallel computation

Tags:, series, python, ipython,     [SUGGEST  A  TAG]
     Series Listing
Part 1: Getting started with IPython on a Windows PC
We discuss effective ways to set up and configure IPython on a Windows PC including handy tips for making the most out of IPython in a Windows environment.
Part 2: Getting around and using the IPython interpreter
We discuss some of the 'magic' commands, how to use the command history, how to use the editor, some useful keystrokes, and how to get interactive help.
Part 3: Using on-line documentation for a short case study on computing integrals
Scipy and numpy are too vast to discuss their constituent components. Instead, we take a simple case study where we want to compute the integral of sampled data and illustrate how to use the on-line documentation effectively.
Part 4: Subclassing and using the Python language for scientific computing
In this segment, we discuss how to subclass an existing object from the scipy module, how to use an external editor with IPython, and how to add customized functions to an existing object
Part 5: Using doctest and docstrings in customized classes
In this segment, we continue with our example of subclassing objects from the scipy module and we discuss how to set up and use docstrings and doctests to provide documentation and testing
Part 6: Python Language Features for Large Data Sets
Here we discuss how generators can be used to process large data sets, which may be too resource-consuming to pull into memory all at once. We further introduce the itertools module as a way of extending the functionality of generators
Part 7: Debugging using IPython
We discuss the following debugging methods: , using %run -d -b , using %pdb to catch errors, using pdb.set_trace() in source code
Part 8: Profiling in IPython
We discuss the following profiling methods:
using %time
using %prun
Part 9: Introducing Sympy -- symbolic computing in Python
Introducing Sympy
-- manipulating expressions
-- defining functions
-- lambdify
Part 10: Getting started with Matplotlib
Matplotlib is one of the best, if not the best, plotting package for scientific Python. In this segment, we discuss how to best use the Matplotlib website to get started with plotting with Matplotlib.
Part 11: Getting Started with Parallel IPython
IPython provides excellent interactive parallel computing tools. In this segment, we discuss how to set up remote computational engines using ipcluster and how to use special magic commands in IPython to use these remote engines.

Here we show how to run the IPython interpreter on the Windows PC (as usual), but connect to and interact with remote computational engines
Part 11a: Basic Parallel Computing Using IPython
In this segment, we discuss cover importing modules for the remote nodes and how to distribute data (i.e. scatter, gather) to and from these nodes.
Part 11b: Distributing Parallel Computations Using IPython
In this segment, we discuss the map feature to distribute computations across the available nodes, how to use targets to specify which nodes to use, and how to use blocking to avoid waiting for long computations to finish.
Part 11c: Finer Control for Parallel Computing in IPython
In this segment, we discuss the push, pull, and push_function, features of IPython that allow finer control over individual namespaces of remote nodes and over the computations that run on them.
Part 12: Case Study -- Robust Regression Methods Using Python
In this segment, we pull together everything we have learned so far to consider robust regression methods that can easily handle the presence of outliers in the data. Here, we discuss what these regression methods are and how we can apply what we have learned so far
Part 12a: Case Study -- Robust Regression Methods Using Python continued
In this segment, we show how to compute the least-median estimate using the sample data set and Python generators.
Part 12b: Case Study -- Robust Regression Methods Using Parallel IPython
In this segment, we show how the intensive least-the in computation can be accelerated using the parallel computing facilities of IPython we have considered so far.
Final: Effective Use of Dictionaries for New Scientific Python Users
In this final segment, we highlight concepts important for new scientific Python programmers, especially those coming from a Matlab environment. We discuss effective use of Python dictionaries and the timeit profiling module. We also discuss the defaultdict object for quick and easy database creation useful for keeping track of data in multiple parameter runs.