Serving IPython notebook on public domain

I’ve been involved in teaching basic programming in Python. There are several good tutorials and on-line courses (just to mention Python@CodeCademy), but I’ve recognised there is a need for some interactive workplace for the students. I’ve got an idea to setup IPython in public domain, as many of the students don’t have Python installed locally or miss certain dependencies…
The task of installing IPython and serving it in publicly seems very easy… But I’ve encountered numerous difficulties on the way, caused by different versions of IPython (ie. split into Jupyter in v4), Apache configuration and firewall setup, just to mention a few. Anyway, I’ve succeeded and I’ve decided to share my experiences here 🙂
First of all, I strongly recommend setting up separate user for serving IPython, as only this way your personal files will be safe 😉

  1. Install IPython notebook and prepare new user
  2. [bash]
    # install python-dev and build essentials
    sudo apt-get install build-essential python-dev

    # install ipython; v3 is recommended
    sudo pip install ipython[all]==3.2.1

    # create new user
    sudo adduser ipython

    # login as new user
    su ipython
    [/bash]

  3. Configure IPython notebook
  4. [bash]
    # create new profile
    ipython profile create nbsever

    # generate pass and checksum
    ipython -c "from IPython.lib import passwd; passwd()"
    # enter your password twice, save it and copy password hash
    ## Out[1]: ‘sha1:[your hashed password here]’

    # add to ~/.ipython/profile_nbserver/ipython_notebook_config.py after `c = get_config()`
    c.NotebookApp.ip = ‘localhost’
    c.NotebookApp.open_browser = False
    c.NotebookApp.port = 8889
    c.NotebookApp.base_url = ‘/ipython’
    c.NotebookApp.password = u’sha1:[your hashed password here]’
    c.NotebookApp.allow_origin=’*’

    # create some directory for notebook files ie. ~/Public/ipython
    mkdir -p ~/Public/ipython
    cd ~/Public/ipython

    # start notebook server
    ipython notebook –profile=nbserver
    [/bash]

  5. Configure Apache2
  6. [bash]
    # enable mods
    sudo a2enmod proxy proxy_http proxy_wstunnel
    sudo service apache2 restart

    # add ipython proxy config to your enabled site ie. /etc/apache2/sites-available/000-default.conf
    # IPython
    <Location "/ipython" >
    ProxyPass http://localhost:8889/ipython
    ProxyPassReverse http://localhost:8889/ipython
    </Location>

    <Location "/ipython/api/kernels/" >
    ProxyPass ws://localhost:8889/ipython/api/kernels/
    ProxyPassReverse ws://localhost:8889/ipython/api/kernels/
    </Location>
    #END

    # restart apache2
    sudo service apache2 restart
    [/bash]

Your public IPython will be accessible at http://yourdomain.com/ipython .
The longest time it took me to realise that c.NotebookApp.allow_origin='*' line is crucial in IPython notebook configuration, otherwise the kernel is loosing connection with an error ‘Connection failed‘ or ‘WebSocket error‘. Additionally, in one of the servers I’ve been trying, there is proxy setup that block some ports high ports, thus it was impossible to connect to WebSocket even with ApacheProxy setup…
If you want to read more especially about setting SSL-enabled notebook, have a look at jupyter documentation.

Advertisements

Apache2 reading from sshfs share

Today, I have encountered problems trying to read data from sshfs share in apache2. I was getting 403 Forbidden error. It turned out you need to enable other_user in sshfs, so other users than the one mounting the share can access the data, as apache2 is using www-data user.
[bash]
# uncomment last line of /etc/fuse.conf
# Allow non-root users to specify the allow_other or allow_root mount options.
user_allow_other

# enable other_user and read access by non-root
sudo chmod a+r /etc/fuse.conf

# remount
sudo umount DESTINATION
sshfs -o allow_other SHARE DESTINATION
[/bash]

Inspired by serverfault and unix.stackexchange.

Change temporary directory in Linux

Sometimes, the size of / (root) mount is limited and in result some processes requiring large /tmp may fail. This can be solved by setting environmental variable TMPDIR:
[bash]
mkdir -p /home/$USER/tmp
TMPDIR=$(mktemp -d /home/$USER/tmp/XXXX)
TMP=$TMPDIR
TEMP=$TMPDIR
export TMPDIR TMP TEMP
[/bash]

More info on serverfault.

Transfer WordPress to Amazon EC2

After rather successful year of using WordPress, I have decided to move my blog to AWS. I was considering the move for long time, motivated by Free Tier and finally I found some time to do it.

At first, I have created WordPress Stack using CloudFormation, but personally I prefer Ubuntu over Amazon Linux and I will focus on configuration of Ubuntu EC2 instance here.

  1. Export your existing blog
    WP-Admin > Tools > Export

  2. Login to AWS console and Create Key Pair
  3. Launch EC2 instance
    I use Ubuntu HVM. I recommend t2.micro, as it’s free for the first year. You should specify created/uploaded key.

  4. Login to your EC2 instance using Public DNS or IP and your key
    [bash]ssh -i .aws/your_key.pem ubuntu@ec2xxxxx.compute.amazonaws.com[/bash]
    NOTE: you key should be readable only by you. To achieve that, you can do:
    [bash]chmod 600 .aws/your_key.pem[/bash]

  5. Configure Ubuntu
    [bash]
    sudo apt-get update && sudo apt-get upgrade
    sudo apt-get install apache2 php5 php5-mysql libapache2-mod-php5 libapache2-mod-auth-mysql mysql-server
    [/bash]

  6. Configure MySQL
    [bash]
    sudo mysql_secure_installation
    mysql -uroot -p

    CREATE DATABASE wordpress;
    CREATE USER ‘wordpress’ IDENTIFIED BY ‘SOMEPASS’;
    GRANT ALL ON wordpress.* TO ‘wordpress’;
    [/bash]

  7. Configure wordpress
    [bash]
    sudo -i
    cd /var/www/html/
    wget https://wordpress.org/latest.tar.gz
    tar xpfz latest.tar.gz
    rm latest.tar.gz
    cd wordpress/
    mv wp-config-sample.php wp-config.php
    sudo chown -R www-data:www-data /var/www/html

    # edit wp-config.php
    define(‘DB_NAME’, ‘wordpress’);
    define(‘DB_USER’, ‘wordpress’);
    define(‘DB_PASSWORD’, ‘SOMEPASS’);
    define(‘DB_HOST’, ‘localhost’);
    [/bash]

  8. Configure Apache
    [bash]
    # edit /etc/apache2/sites-available/wordpress.conf

    ServerName ec2xxxxx.compute.amazonaws.com
    ServerAlias YOURDOMAIN.COM
    DocumentRoot /var/www/html/wordpress
    DirectoryIndex index.php

    AllowOverride All
    Order Deny,Allow
    Allow from all

    # enable wordpress in apache2
    sudo a2ensite wordpress
    sudo service apache2 restart
    [/bash]

  9. Enable HTTP access to your EC2 instance
    Go to EC2 console > Instances > Select you instance > Description >
    Click on your `Security group` > Select Inbound > Edit > Add rule > HTTP > Save

  10. Point your webrowser to your EC2 instance: http://ec2xxxxx.compute.amazonaws.com/
  11. Setup your wordpress account
  12. Upload dumped wordpress data
    WP-Admin > Tools > Import > WordPress > > Upload file import
    NOTES:
    You will need to install WordPress Importer plugin.

  13. Assign post to correct user.
    Don’t forget to Import Attachments!

  14. Install your favourite plugins and themes
    As for plugins, I strongly recommend: JetPack, SyntaxHighlighter Evolved and Google Analytics Dashboard for WP.

  15. Add favicon
    Copy selected favicon.ico to /var/www/html/wordpress

Voilà!
BTW: You may want to increase security of your instance and setup swap just in case memory usage exceeds your EC2 instance size.

EC2 instance safety instructions

  1. Add non-default user and add it to sudo group
    [bash]
    sudo adduser USERNAME
    sudo usermod -a -G sudo USERNAME
    # switch user
    su USERNAME
    [/bash]

  2. Edit /etc/ssh/sshd_config
    [bash]
    # change port to non-default port ie 3434
    # & add this port to your instance Security Groups > Inbound
    Port 3434

    # enable password authentication
    PasswordAuthentication yes

    # restart ssh
    sudo service ssh restart

    ###
    # make sure you can login with
    # your new username before continuing
    ###

    # disable root login without password by commenting:
    #PermitRootLogin without-password

    # restart ssh
    sudo service ssh restart
    [/bash]

  3. Secure MySQL isntallation
    [bash]sudo mysql_secure_installation[/bash]

  4. Reboot
    [bash]sudo reboot[/bash]