Keyboard Maestro macro to open .pickle file in IPython

I often want to view the contents of .pickle files on my computer. I like doing this in the IPython REPL, so I would manually open a new Terminal window, switch to my data science virtualenv, import pickle, and then load the .pickle file in question into a variable. Sounds like a prime target for automation, eh?

Here's a Keyboard Maestro macro that will take the selected .pickle file in Finder and open it up in IPython (inside iTerm2), assigning it to the variable x.

Here's what it looks like (the macro is triggered through LaunchBar with this custom action):

After the macro runs, I type in x to show the object in the Pickle file (in this case, it's just {"hello": "world"}.

Here's what the macro looks like in Keyboard Maestro:

Here's the code:

Note that this requires your Python virtualenv to be set up like this and you have to have a "data" virtualenv. If either of these things aren't true, you'll need to modify the shell commands in the AppleScript accordingly.

IPython Notebook Setup Code

I just posted the code I use at the top of every IPython notebook to GitHub: https://github.com/masnick/ipython-setup

It was somewhat difficult to figure out what to import (and the conventions for this) when starting with IPython. This setup file takes care of that.

It also has some additional goodies like my code toggle button and my method to display raw HTML inline in the notebook.

In any case, I recommend standardizing the contents of the first cell in IPython notebooks, saving this out to a file, and then using %load magic to bootstrap the first cell. (This is explained in more detail in the GitHub link above.)

SciPy: Using GridSpec to Display Box Plots Under Histograms

Here's how to get output like this using matplotlib and its GridSpec class:


import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import matplotlib.gridspec as gs

%matplotlib inline

df = pd.DataFrame([
        np.random.randn(1000),
        np.random.random_integers(1, 2, 1000)
    ]).T
df.columns = ["value", "group"]


fig = plt.figure(figsize=(10, 6))
grid = gs.GridSpec(2, 2, height_ratios=[3, 1]) 
for i, g in enumerate([1, 2]):
    
    subset = df['value'][df['group'] == g]
    
    # Histogram
    ax = plt.subplot(grid[i])
    ax.hist(list(subset), color='k', alpha=0.4)
    ax.set_title("Group %s" % g)
    
    # Box plot
    ax2 = plt.subplot(grid[i+2])
    pd.DataFrame(subset).boxplot(vert=False, return_type='axes')
    ax2.set_yticklabels([''])

fig.tight_layout()
fig.subplots_adjust(top=0.85)
fig.suptitle('Example', fontsize=20)

None # Don't display the last thing -- `%matplotlib inline` will display the graphs no matter what.

It looks like Gist embed is broken here on PostHaven for IPython notebooks, so the embed below may look strange:

IPython and Jupyter Notebooks: Automatically Export .py and .html

[Updated 2016-03-04 to support Jupyter 4 notebooks – see below.]

IPython notebooks are stored in a format that is not particularly human-readable and doesn’t work well in version control.

One way to solve this problem is to automatically export the code from IPython notebooks into a vanilla Python file after each save.

It’s also useful to automatically generate a HTML file of the notebook on each save. This can be done manually in Jupyter (File > Download as > HTML), but if you always want this, doing it automatically is much easier.

Use the following code to automatically save a .py and a .html file when you save a notebook in Jupyter. These two files will be saved in the same folder as the parent .ipynb file.

First, run ipython locate profile default, which will give you the path to save the following code in.

Save the code below in this folder as ipython_notebook_config.py:

Now, run ipython notebook. You will see an error message in the terminal if there are any syntax or runtime errors with ipython_notebook_config.py. If everything looks good, go to your web browser, open a notebook, and click the Save/Checkpoint button (it’s the floppy disk icon in the Jupyter toolbar). You should see a .py and a .html file appear alongside your .ipynb file.

Update for Jupyter 4 notebooks

After the big split between IPython and Jupyter, and the accompanying update to Jupyter 4.x, the config file above no longer works. Instead, put the following in a file saved at ~/.jupyter/jupyter_notebook_config.py to achieve the same thing:

This code could be DRYer, but it does work.

Hide Code When Sharing IPython Notebooks

When sharing IPython notebooks with non-programmers, I don't want to cause confusion by including a bunch of code.

To this end, I wrote a snippet of Python and JavaScript that can be put in a code cell at the top of a notebook. When a notebook with this code is exported as HTML or viewed with nbviewer.ipython.org, code blocks will be hidden by default. A "Toggle code" button is also added in case you want to see the hidden code.

Here is the code snippet to put in the cell at the top of the notebook:

And here is an example of how this works in nbviewer.ipython.org.

(The source code for this example and the accompanying HTML output are available here.)