Raspberry Pi Hardware

First of all, we need a Raspberry Pi. The newest model is Raspberry Pi Model 3B, which is what I am using. Here is the specification:

  • A 1.2 GHz 64-bit Quad-core ARMv8 CPU
  • 1GB Memory
  • 100Mbps Ethernet Port
  • 802.11n Wifi
  • Bluetooth 4.1 BLE
  • 4 USB ports
  • Micro SD card slot
  • HDMI output
  • 3.5mm audio output
  • Micro USB power input
  • 40-pin GPIO
  • CPI (Camera) input

Note that there is no hard drive in Raspberry Pi, everything is stored in the Micro SD card. A 8GB Class 4 Micro SD card is sufficient. However for the best performance, a 16GB class 10 Micro SD card is recommended.

Also, a power supply is required. The official power supply is 5V 2.5A Micro USB. Most of the Android mobile chargers are 5V 2A and sufficient.

Last but not least, there are a few options for initial connect and setup. A keyboard (wired or wireless, but not bluetooth) and a monitor (and a HDMI cable) can be used, or simply an ethernet cable connect to the router.

You may also want a case to protect you Raspberry Pi. You can buy one online, or print one with 3D printer. Nevertheless, you can always make one with card board.

Also, a low end Raspberry Pi Zero is available. I recently purchased a Zero W, W for wireless. It has a weaker processor and a lot less connectors, and much smaller in size and cheaper.

Here is what Pi Zero has:

  • A 1 GHz 32-bit ARMv6Z CPU
  • 512MB Memory
  • Micro SD card slot
  • Mini-HDMI output
  • Micro USB power input
  • On-the-go Micro USB (require adaptor)
  • 40-pin GPIO
  • Mini-CPI (Camera) input

In addition, Pi Zero W has a chip for wifi and bluetooth:

  • 802.11n Wifi
  • Bluetooth 4.1 BLE

Any one of these is all we need for hardware. Let’s start our system.

Install the System

There are multiple systems available for Raspberry Pi. Most of them are based on Linux. The official system is Raspbian, a Debian based system designed for Raspberry Pi. The latest disk image is available at https://www.raspberrypi.org/downloads/raspbian.

After download the disk image, it needs to be written into the SD card. Note, the SD card will be formatted and all data will be removed. Please back up your data before continue.

For Windows, Win32 Disk Imager can be downloaded at https://sourceforge.net/projects/win32diskimager.

Insert your SD card, run Win32 Disk Imager as administrator, select the disk image as the source file and your SD card as destination. Click on write, then wait for the writing to complete.

For MacOS, no additional software is required.

First, insert your SD card, open Disk utility. Choose your SD card from the left. The table on the right indicates the device number of “disk1” (or any other number). Record that number. Then unmount the disk (on the top row).

Second, open Terminal. Type in the following command:

  • sudo dd if=<path of raspbian.img> of=/dev/disk<number>

Where the input raspbian.img is the location of your disk image (usually at ~/Downloads/<date>raspbian.img) and the output disk number is the one recorded on the previous step. Hit Enter, and type in your administrator password, then wait for the writing to complete.

Hint: You can type only the first few characters of the image file name, after its directory, and hit on tab to auto select. When you enter your password, unlike when you login to the system, nothing will be shown for security propose. During writing process, you can use control+T to see its progress.

The writing process takes about 30-40 minutes for the full version of system (about 4GB). Eject the SD card and it’s all done!

The First Sight of Raspbian

After the system is installed, insert the SD card to the Raspberry Pi, connect other devices such as mouse, keyboard, monitor and Ethernet cable, we’re ready to boot by plug in the power supply.

There is no on/off button on Raspberry Pi, unlike traditional PC. The Raspberry Pi will automatically boot once it’s connected to power.

After a long booting screen, the command line is ready for us. Raspberry Pi has graphical user interface (Desktop), but it’s not turned on by default.This tutorial is focus on command line, however we can enable desktop by type in “startx”, or we can use raspi-config to allow desktop starts automatically.

If you don’t have a monitor and would like to remain headless, you can access Raspberry Pi by SSH. This is discussed in the later post.

Raspi-config is a basic config tool on Raspbian, and it can be accessed by command:

  • sudo raspi-config

A new screen will pop out. Move the cursor by arrow and select by enter. There are a few things we must configured for first time use.

First, we need to expand file system. Simply hit on Option 1 and wait for the scrolling. This will allow all space on the SD card is allocated to the system.

Second, change the password on Option 2. Again as you type in your password nothing will be shown, but it will be recorded.

Third, go to Option 4, and change all settings under it to your own locale and timezone, except for keyboard layout.

If you’re using Raspberry pi 1 or 2, also turn overclock on to 1GHz, that’s Option 6 – Select Turbo. If you’re on Pi 3 or Pi Zero, overclock is not available as it’s already fast enough.

After these steps, hit on tab and choose finish, then reboot.

We’ve done on basic setup, and moving forward to softwares.

Basic Commands of Debian (Linux)

As we’re working on command line, there are a lot of basic and useful commands that we need to know about. Please note that all commands are case sensitive.

sudo <command>

sudo means execute the following command as root user, which is equivalent to run as administrator in Windows. Root user is the user with the top privilege in linux. There are many commands that can only be executed by root user, the editing of many files also requires root privilege.

If you execute “sudo su”, you’re switching to root user and all commands are executed as root. You can exit root user by command “exit”.

halt

halt is the same as shutdown. This command requires sudo. To boot Raspberry Pi again, unplug the power and plug it back in.

reboot

reboot is the same as restart. This command also requires sudo.

man <command>

man is short for manual. It can show all usages and options and the explanation of a specific command.

Also, “-h” or “–help” option of a command would normally output a short summary of the manual.

Files and Directories

ls <path>

ls is short for list files, under current directory by default if no path is given. In Linux, file names are case sensitive. Also a directory is considered as a file, most files don’t have a extension.

Linux treats all files and directories have the name starts with . (a single dot) as hidden files/directories. If you would like to see the hidden files, use “ls -a”.

cd <path>

cd is short for change directory. We’re moving from one directory to another. Path can be type in absolute path (full path) or relative path (under current directory). If the path starts with /, it’s considered to be an absolute path. Otherwise it’s a relative path.

For long directory name, you can type in the first few characters, and hit tab, the system would automatically fill out the rest for you. If there are multiple matches, all matches will be listed and you can type in the one you want. This also works on flies name and some commands.

There are a few more expressions about path and directory in linux:

  • (a single dot) represents current directory.
  • .. (two dots) represents the parent directory of the current directory.
  • / (forward slash) represents the top directory. It doesn’t have a parent directory.
  • ~ (tilde) represents the home directory of the current user.

pwd

pwd is short for print working directory. It will output the absolute path of your current location.

mkdir <path>

mkdir is short for make directory. The path can be either absolute path or relative path. However you cannot make a directory that has the same name with another file or directory, under the same parent directory.

rmdir <path>

rmdir is short for remove directory. Again the path can be absolute of relative. Unlike other systems, you can only do rmdir on empty directory. If there are other files in the directory and you would like to delete them all, use “rm -r”.

cp <source files> <destination path>

cp means copy. Copy the source files to the destination. If you would like to copy a directory, use “cp -r”.

Multiple source files are accepted, and the last input is always the destination. * (wildcard) can also be used to select all matched files.

mv <source files> <destination path>

mv means move. Move the source files to the destination. If you would like to move a directory, use “mv -r”.

rm <files>

rm means remove files, or delete files. To delete a directory, use “rm -r”. This works on a empty directory and  a directory contains files. Note that there’s no trash on command line and all delete are permanent and cannot undo.

chown <owner:group> <files>

chown means change owner. Each file has an owner and a file group, as well as access permission control. Change the owner of a file would allow the new owner takes control of the file. root user can usually override the access control settings and change any files.

If you change the owner for a directory, it would only apply to the directory itself. If you would also like to change the owner for all files under it, use “chown -r”.

chmod <permission> <files>

chmod means change access permission. Access permission is a three digit number and each digit is range 0-7. The first digit is the owner’s permission. Second digit is the file group users’ permission. And the last digit represents all other users’ permission. There are three permissions, read permission is notated by number 4, write permission is number 2 and execute permission is number 1. Where multiple permission is gained, the corresponded numbers add up.

If you change access permission for a directory, it would only apply to the directory itself. If you would also like to change the access permission to all files under it, use “chmod -r”.

-r/-R option

-r/-R means recursive.  Recursion means to execute the command not only to the directory itself, but also to all files under it. Some commands require capital and some others use lowercase.

Read and Edit Files

cat <files>

cat is to output the content of a file, or read the file.

head <files>

head is similar to cat. However it outputs only the first few lines of the file.

tail <files>

tail is also similar to cat, but it outputs the bottom few lines only.

grep <pattern> <files>

grep is to search for a specific string (pattern) in the input file. Only the lines that contain given string will be outputted.

nano <files>

nano is a basic and simple editor for binary files. It can either edit an existing file or create a new file if the input file doesn’t exist. The ^ symbol for keyboard command is control key.

There are also other editors pre-installed, such as vim. But nano is always the easiest to use.

 

These are the basic commands we’re interested in at the moment. As we explore, we will be using more command to speed up our work. We can now start to work on softwares and packages.

 

Softwares on Raspberry Pi

Softwares are often referred as packages in Linux. There are a few ways to install new packages. Some packages can be downloaded on websites, but most are done by apt-get. Many popular packages are listed on the source lists and can be downloaded directly over command line. apt-get can also update existing packages and deal with dependency.

Right after install the system, it’s always good to check any updates available. This is also recommended before you install any packages for the first time of the day. The update command for all apt-get packages is:

  • sudo apt-get update
  • sudo apt-get upgrade

Note that these two commands are not the same. Update is to update the source list only (check for newer version) and upgrade is the actual installation of newer version of packages. Both lines need to be executed and in order, i.e. first update then upgrade.

However, upgrade will not deal with dependency, so sometimes it would say a few packages are not upgraded. This means there are new packages require to install before upgrade existing ones. If that’s the case and you really want to upgrade, use dist-upgrade:

  • sudo apt-get dist-upgrade

To install new packages, execute the following command:

  • sudo apt-get install <package name>

And to uninstall packages, execute the following command:

  • sudo apt-get remove <package name>

Make sure the package name is as exact or you will receive an error. If you’re not sure, type in the first few characters and hit tab to see all matches.

Apart from apt-get, curl and wget are sometimes used to install as well. They are more like a command line downloader. For the use of curl and wget, it’s better to copy the command from the developer’s website to avoid errors.

Vim – An Advance Text Editor

You may already get tired of the nano text editor. It’s simple but not efficient nor sufficient. So we are introducing Vim – Vi Improve.

Vim is a powerful and highly customisable text editor. It’s based on and an improvement of Vi, another text editor included in Raspbian. The trade off for power is of course, the cost to learn. So I didn’t put this as a basic level of the tutorial, but as an advance.

To install vim, like all other packages:

  • sudo apt-get install vim

To start:

  • (sudo) vim <filename>

Like nano, if file exist, vim will open that file, otherwise a new file will be created.

Directories are also a special file in Linux. However nano cannot open them but vim can. you can try to open a directory and see what happens. It acts as a document tree!

Now open a blank file and we’ll go though some basic of vim:

  • vim ~/test

At the bottom we can see the name of file and it’s a new file, then two numbers 0,0-1, that’s the position of the cursor (line number 0, Position in the line 0). We ignore the All at the moment.

Vim as two modes, command mode and insert mode. The default is command mode which treat all keyboard input as commands. To enter insert mode to make direct changes, we need to switch to insert mode first by hitting i.

Now INSERT should be appear at the lower left and all inputs are actually recording.  Exit insert mode by hitting esc, and the INSERT would disappear, means vim is switched back to command mode.

To save the file and exit vim, enter :wq under command mode.

Here are more commands that are often used, all command are used in command mode only and they are case sensitive:

Enter Insert Mode:

i – insert before the cursor

a – append, insert after the cursor

I – insert at the start of the line

A – Append, Insert after the end of the line

o – start a new line below and insert

O – start a new line above and insert

s – delete the cursor character and insert

S – delete the line and insert

r – replace cursor character

R – replace mode (insert mode but replace)

Move the Cursor and Navigate:

In insert mode, arrows are the only way to navigate, but there are a lot more ways in command mode apart from the arrows. Arrows can be repeated for a number of times (jump) if there’s a number before it, eg. 3h – move 3 characters to the left.

h – left arrow

j – up arrow

k – down arrow

l – right arrow

gg – go to start of the file

G – go to end of the file

<Line Number> G – go to line

0 (zero) – go to start of line

$ – go to end of line

% – find match bracket

. (dot) – repeat last command

Cut, Copy, Paste and Delete:

In vim, all deletes act as cut, saved to a buffer and can be paste later. Most of these commands can be repeated for a number of times if there’s a number before the command, eg 12x – delete next 12 characters, 3Y – copy next 3 lines.

x – delete (cut) the character under the cursor

X – delete (cut) the character before the cursor

d<h/l> – delete (cut) the character under/before the cursor, equivalent to x/X.

d<j/k> – delete (cut) the line above/below.

dd – delete (cut) the current line

y<h/l> – yank (copy) a character before/under the cursor

y<j/k> – yank (copy) the line above/below

Y – yank (copy) the current line

p – paste after the cursor (or new line below)

P – paster before the cursor (or now line above)

Undo and Redo:

u – undo

U – undo current line only

ctrl + r – redo

Find and Replace:

Find and Replace are also case sensitive. If a special character need to be enter, have a \ before it, eg. to find all !=, enter /\!\= instead of /!=

/<string> – find <string> (from top to bottom)

?<string> – find <string> in reverse order (from bottom to top)

n – next match

N – previous match

:s/<old>/<new>/g – substitute  <old> with <new> in the current line

:%s/<old>/<new>/g – substitute  <old> with <new> in the whole file

Open, Save, Quit:

These commands can be combined. Start with : (column).

:w – save file

:w <filename> – save as <filename>

:o/:e <filename> – open/new file

:q – quit (exit) vim

! – override warnings (ignore all warnings, normally used with :q! – quit without saving)

:! <command> – execute command using system shell

:n – next file (if open multiple files in shell)

:N – previous file

:help – help file

Here’s a cheatsheet for all commands from GitHub: Vim Cheat Sheet

 

Apart from these powerful commands, vim setting file also make an important role:

  • vim ~/.vimrc

As a beginner, there lines are recommended:

  • syntax on
  • set nocompatible
  • tabstop=4
  • shiftwidth=4
  • set autoindent
  • set smartindent

If you prefer line number:

  • set number

If you want to use your mouse pointer to navigate:

  • set mouse=a

Save and exit, and open vim again to see changes.

You may seek more vimrc command to build your personal vim.

Last but not least, vim provides many useful plugins that are awaiting for you to discover!

General Knowledge of Networking

 

On Internet model, there are 5 layers. They’re Physical Layer, Data Link Layer, Network Layer, Transport Layer and Application Layer. Each of them has its own protocols and addresses.  Each layer can only communicates with its adjacent layers.

Addresses

MAC Address is the address in Data Link Layer (Layer 2). IP Address is the address in Network Layer (Layer 3). Port Number is the address in Transport Layer (Layer 4). Mostly, we’re dealing with IP address and Port number.

IP Address

IP address is used to identify a host. IPv4 address is ranged from 0.0.0.0-255.255.255.255. Each section is from 0-255. An IP address can be private, public or for special use.

Private IP address can be used in local network only, it doesn’t resolve on the internet. LAN IP are often private IP. Private IP are ranged at 10.0.0.0-10.255.255.255, 172.16.0.0-172.31.255.255, 192.168.0.0-192.168.255.255.

Public IP address are used on the internet, WAN IP are often public IP. Public IP is ranged from 1.0.0.0-223.255.255, except for Private IP, 127.0.0.0-127.255.255.255 (loopback address) and 169.254.0.0-169.254.255.255 (link local address).

The remaining addresses are reserved or for special use.

Port Number

Port number indicates which application the packet is sending to. Port number can be ranged from 0-65535. Port number on server side is often static and on client side it’s dynamic. Port 49151-65535 are explicitly for private use (client port).

Well known server ports includes 21 (FTP), 22 (SSH), 23 (Telnet), 25 (SMTP), 53 (DNS), 67/68 (DHCP), 80 (HTTP), 110 (POP3), 137/139 (Samba), 143 (IMAP), 443 (HTTPS), 465/587 (SMTPS), 520 (RIP), 546/547 (DHCPv6), 993 (IMAPS), 995 (POP3S).

NAT

Network Address Translation is widely used in home router. It allows us to use the same public IP address (the one from ISP) with multiple devices. The IP address of our devices is actually a private address, which means it can be used in LAN only.

Let’s see what’s happening when we browse a webpage in LAN:

  • When we open Chrome, the computer assigns a random port number. If we go to google from Chrome, Chrome sends a packet to Google with source destination and its assigned port (Let’s say 192.168.0.3:54321 -> 216.58.220.110:443).
  • Then the router maps the private address and source port to a random port in router, then alters the source address and source port of the packet (e.g. 123.234.321.23:11111) and send it to Google, now the packet becomes 123.234.321.23:11111 -> 216.58.220.110:443.
  • Google replies to the router with the webpage, it’s destination is 123.234.321.23:11111, the router maps back to the original private address and rewrites the destination to 192.168.0.3:54321. Then your computer knows port 54321 is for Chrome and sent the data to it.

That’s it for what we need to configure our device working on LAN. With the basic concept of how network works we can now proceed to implement access to Raspberry Pi.

Connect to Wifi

The Raspberry Pi Model 3B has built-in 802.11n wireless network card, which means you can connect to internet over wifi. For all other model, you may purchase a USB wifi dongle.

The wifi network is controlled by wpa_supplicant, edit the file with the following command:

  • sudo nano /etc/wpa_supplicant/wpa_supplicant.conf

Enter the network detail as the following format at the end of the file:

  • network={
  •     ssid=”<wifi SSID>”
  •     psk=”<wifi Password>”
  • }

Note that the quotation marks are compulsory. Save and exit nano, then restart wireless service by the following command:

  • sudo ifdown wlan0
  • sudo ifup wlan0

You should be see the IP address on “ifconfig wlan0” under inet address. If not, check your SSID and password.

 

Static IP Address

As the device asks DHCP server for IP address when it connects, the IP address changes over time. Static IP allows the device reserves an IP address and doesn’t change over time. It is not compulsory but highly recommended. A static address allows quick access if you would like to setup remote access in LAN, and it’s essential if you would like to run a server and allow access from internet via NAT.

Static IP can be configured on router or on Raspberry Pi. To eliminate the possibility of IP address conflict, configure on router is recommended.

To configure on router:

First, check your MAC address by the following command:

  • ifconfig <interface>

If you’re using ethernet cable, the interface would be “eth0”.

If you’re connected to wifi, the interface would be “wlan0”.

The HWaddr is the MAC address and the inet addr is the IP address we need, record these addresses.

Second, login to your router, look for static IP settings. It’s usually under or close to DHCP Server settings. Enter the MAC address and IP address recorded before and save. If you like (and you know how it works), you may also bound the device to another IP address.

This IP address now permanently belongs to your pi on your network. Reboot the device to make it in effect.

To configure on Raspberry Pi:

First, check your default gateway and DNS settings by the following commands:

  • ifconfig <interface>
  • route

If you’re using ethernet cable, the interface would be “eth0”.

If you’re connected to wifi, the interface would be “wlan0”.

For the first command, The inet addr is the device IP address and mask is subnet mask we need, record these address. We also need the Bcast address.

For the second command, the gateway address for default destination is the IP address of the router, we need these as well.

Second, edit the interface file by command:

  • sudo nano /etc/network/interface

Find the appropriate interface, Enter the detail as the following format under it:

  • iface <interface> inet manual static
  •     address <IP address of Raspberry Pi>
  •     netmask <subnet mask>
  •     gateway <IP address of router>
  •     network <Bcast address, replace the last 255 with 0>
  •     broadcast <Bcast address>

Originally, The last word of the first line is manual and we need to replace it with static. You may also use another IP address if you like (and know what you’re doing).

Third, we also need to update DNS address on resolv.conf:

  • sudo nano /etc/resolv.conf

Enter the detail as the following format at the end:

  • nameserver <IP address of router>

If you’re using another DNS server, you can use the one you prefer.

Note that this method may result in IP conflict, therefore it’s not recommended. Reboot your Raspberry Pi to make them in effect.

Remote Login – SSH

Raspberry Pi can be accessed remotely by Secure Shell (SSH) by default (Default to be off since Sep 2016 release). You can connect your PC to Raspberry Pi via SSH and use the command line just the same way as you’re on the Pi. To use SSH, you need the IP address of the Raspberry Pi (that’s why static IP is recommended), and your username and password, or you can create a certificate later and use it instead of a password.

Other than the IP address, Raspberry Pi can also be connected by hostname. The default hostname is “raspberrypi.local”. If you don’t know the IP address, this is a replacement of it.

The default username is “pi” and the default password is “raspberry”. If you have changed your password on earlier setup (and you should), use your own password.

On Windows, PuTTY is the the most popular client for remote access. It can be downloaded at http://www.putty.org.

Run PuTTY as administrator, select SSH for connection type, enter the IP address and login information, then connect.

On Mac OS, we use terminal to connect to Raspberry Pi. Type in the one of following command:

  • ssh <username>@<IP address>

Then type in your password and get connected.

SSH can also be used to transfer files between Raspberry Pi and your PC. This is called Secure Copy (SCP).

On Windows, you need to download PSCP from PuTTY ebsites . Open Command Line and use the following command:

  • If you want to copy file from your PC to Raspberry Pi:
    • pscp <source files> <username>@<IP address>:<destination path>
  • If you want to copy file from your Raspberry Pi to your PC:
    • pscp <username>@<IP address>:<source files> <destination path>

On Mac OS, scp is integrated in terminal. The command is as following:

  • If you want to copy file from your Mac to Raspberry Pi:
    • scp <source files> <username>@<IP address>:<destination path>
  • If you want to copy file from your Raspberry Pi to your Mac:
    • scp <username>@<IP address>:<source files> <destination path>

We also need to disable root login via SSH, especially if you would like to setup access via internet, as root password cannot be change. This is configured by sshd_config, edit the file with the following command:

  • sudo nano /etc/ssh/sshd_config

Find the following line:

  • PermitRootLogin yes no

Change from yes to no to disable root login. Then save and exit, and restart SSH service:

  • sudo service ssh restart

If you plan to make a headless pi and access via SSH all the time, SSH is off as default from Sep 2016. You may create an empty file named ssh under boot partition to override this setting after install the system to SD card.

With SSH, we no longer need a monitor and a keyboard to access Raspberry Pi. It’s a huge save of money for an extra monitor, Hurray!