Using Testdisk to recover lost files from deleted partitions

I installed ubuntu on my home desktop recently. While doing so I choose the “Replace Windows …” option which like title suggests, should have just replaced the windows partition with a ubuntu partition and should have kept everything intact (or so I thought). However, in reality the option is misleading and is actually used to format the whole drive and use all of it for linux. It is like doing a fresh single boot install of ubuntu on your machine. I would highly advise against using this option even though it is the second suggested option (issue with user experience).

In my 500 GB HDD I had 4 partitions:

• C:\ – Windows, ~50 GB
• D:\ – Personal files, ~200 GB
• E:\ – Personal files, ~195 GB
• F:\ – Backup files ~ Rest

I wanted to just replace Windows and instead have Ubuntu on my C:\ drive. Also, I wanted to keep all my data in the other partitions. However, using the “Replace Windows…” option was disastarous and I ended up with one large linux partition of size ~490GB. I spent the rest of the day, trying to recover parts of my original data.

This is when I found out about testdisk, a utility for recovering files and lost partitions from a device. The utility is suggested in many threads related to solving the exact same problem I was having [1,2,3]. After digging through the step by step wiki as well as going through many other links [4,5,6] I recovered many important files from my partitions. This video guide was also very helpful.

Some important things to note while doing the recovery are:

1. First go through the following links in detail: http://selfsolve.net/software/testdisk.html and step by step wiki
2. DO NOT USE the HDD once you realize the partitions were removed or corrupted.
3. Have another external storage device like HDD or USB stick which can store the data you want to recover.
4. Have a Live Ubuntu USB Image. [You will need to boot using this and not use the HDD of your computer. ]
5. Have an internet connection. [Or else you need to have an executable of testdisk with you]
6. Boot into the live ubuntu and open software and updates, and make sure the “Universe” repository is checked. [3]
7. Run the following commands “sudo apt-get update” and “sudo apt-get install testdisk”
8. Once testdisk is installed you need to run it as administrator “sudo testdisk”
9. Follow the steps in
10. Quick search might not yeild best results especially if your Ubuntu was installed over your windows partitions.
11. Try Deeper Search [Be warned this will take a lot of time depending on your HDD size, mine took around 1.5 Hrs]
12. Once finished it should probably list your original partitions usually with first column of results as HDFS – NTFS.
13. You can go inside each partition and look at the files using option “P”. You can also mark paritions for deletion or as Logical Drive using the Left/Right arrow keys.

I used the list files option and copied all important files from my partitions to my external HDD. This will take some time depending on the data you need. I recovered ~5000 photos and few 100 documents and ~5000 other files. The process was easy and the external HDD had the correct files stored.

My attempt to recover full partitions, failed with Invalid partition sectors error. However, I was happy with the capabilities of the tool and its user friendly approach. I believe, it would have been possible to recover my partitions as well but I made some mistake on my side.

Overall, testdisk is a great utility and is recommended for recoveing files which you accidently deleted or lost during partition deletion. It is recommended by the Ubuntu data recovery page as well.

Th

Flying Hubsan X4 using Arduino

This blog post is about a recent class project I have been doing with my team mates for the DIY Intel Galileo class. The main aim of the project was to replicate the already existing projects on flying a Hubsan X4 using Arduino on an intel galileo gen 2 board.

The RX/TX circuit design was built using A7105 Trans receiver.

Our circuit connections using A7105 with Arduino pins. Pin numbers are for Arduino.

In the process of doing so we ended up first trying out using the available resources on an Arduino Uno board. The details of our project and how we proceeded using some existing libraries and circuit design are mentioned on our wiki page at: https://github.com/napsternxg/coptermanager-arduino/wiki

Phase 1:

Getting the circuit to work with the code using Arduino Uno. I had to modify a bit of the excellent library written andihit at by https://github.com/andihit/coptermanager-arduino

Here is the demo:

Phase 2:

This phase included getting the full fledged code working and we demoed it in our class. Here is the footage.

Phase 3:

Using the https://github.com/andihit/coptermanager-simple library we have implemented the keyboard control solution for controlling the quad.

Issues with Intel Galileo Gen 2:

We wanted to port this same code and circuit on the Intel Galileo Gen 2 using its Arduino Pins.

However we faced a major issue in the SPI pin configuration. Pin 11 on the Galileo was not acting the same way as Pin 11 on the Arduino. We checked the voltage across major pins on both arduino and galileo and found the difference only on Pin 11 which is connected to the SDIO pin on A7105. The voltage on Pin 11 is 0 when using on a galileo but on arduino it gives a good 3.79 volts. We have detailed all that info on our Wiki page mentioned above.

Thanks a lot to my team mates Ishita and Abhinav for their effort in making this project a reality. Hopefully we will be able to get this running on galileo with some tweaks.

Intel Galileo Gen 2 Board for interfacing internet of things: Controlling LED Via Web

If you are following this blog series, I hope you are comfortable with running a big Linux image on your Galileo as explained in my previous post.

In this post we will be using the Intel Galileo board to run a web server which will serve a page for controlling an LED connected to a board. This means that if your board is on a network any user can access the page served by the board and switch the LED on or off. The implications and extensions of this example app can be for controlling any device connected to the board via the web.

In this post we will work using a different linux image called the Internet of Things version. This image comes with many plugins for interfacing the with the GPIO pins and running web servers. You can download this new image from the Intel IoT downloads page. More details on  this image and its usage can be found at the intel pages. Also, I would like to point out that the steps for installing this image on the SD card are quite straightforward but also different from installing the default image I mentioned in my older article and require downloading of additional software. You can find detailed instructions for installing this image for windows platform on the intel page.

Once the image is installed you can connect your device to your router and have it available on your network. Now you can run a web server directly on the image using python or the LightHTTP package which comes with it. However, the Inter IoT community has released an XDK which I will be using in this post to configure the web server and send signals to my GPIO pins to switch the LED on/off. The XDK is for running Node.js code on the board and comes with sample code for interfacing with the pins.

The XDK will detect the board if it is present on the network and will deploy the code. It has easy options for starting the deployed app and stopping it all without physically connecting to the Galileo. The editing experience on it is a bit clunky but it serves the purpose of file deployment. I edited my main code in another editor and used the XDK only for deployment.

I used the IoT local temperature example as a template and modified its code to configure the Node.js server and the board interaction.

The app requires 3 important components and all the code is available at: https://github.com/napsternxg/IntelGalileoIoT_LEDOnOff

1. LED circuit connected to Galileo
2. Web page with On/Off switch served by the Galileo
3. Sending the input from web page to the pins.

The final product looks like the following images (click image for more pics):

LED circuit with Intel Galileo

 

In my web app I have used the Socket.io plugin for sending and receiving the signals asynchronously. The main component of the code for interfacing with the GPIO pins is the one using mraa plugin.

var mraa = require('mraa'); //require mraa
console.log('MRAA Version: ' + mraa.getVersion()); //write the mraa version to the Intel XDK console
var myOnboardLed = new mraa.Gpio(13); //LED hooked up to digital pin 13 (or built in pin on Galileo Gen1 & Gen2)
myOnboardLed.dir(mraa.DIR_OUT); //set the gpio direction to output
var ledState = false; //Boolean to hold the state of Led

After initializing the pins we just need to use the following piece of code to switch the LED on/off.

myOnboardLed.write(1); //Send High leading to LED ON
myOnboardLed.write(0); //Send Low leading to LED OFF

LED On Off Web Interface

Overall this was a major step for me in understanding the concept of internet of things and getting my first internet of thing (simple LED) to be controlled over the internet at the flick of a button.

Calculating Pi using Monte Carlo Simulation

Recently I have been intrigued by the uses of Monte Carlo simulation and a myriad of their uses in probabilistic modelling techniques especially in NLP and social network modelling. I wanted to learn more about the monte carlo process and understand what are some of its basic uses. An often cited example people gave me was that of calculating the value of $\pi$ defining it as 4 times the area of a circle inscribed in a unit length square.

The below figure shows the distribution of randomly generated points falling inside and outside the circle. Also the other figure shows how with the greater number of iterations the value of pi converges to the actual value of pi.

Monte Carlo Method for approximating the value of pi using ratio of area of circle inscribed in unit length square.

The intuition behind the simulation process is that if we randomly generate points confined inside a square circumscribing a circle then the probability of the points falling inside the circle will be directly proportional to the area of the circle. Since the circle is inscribed in the square hence its radius is 0.5 and hence the area will be $\pi/4$. This ratio is also approximately equal to the number points randomly generated inside the square which fall inside the circle to the total generated points. The approximation will should converge to the value equal to pi if we have very large number of points ideally tending to infinite points inside the square.

The below code does the simulation and generates the above figures:

	"""
	Using Monte Carlo to find the value of pi.
	Intiution: Ratio of area of circle to area of uniq length squre it is inscribed it can be used to approximate pi.
	Area of circle = pi/4
	Area of square = 1
	Ratio = pi/4
	Let us generate random points x,y inside the square. The probabilty of the point being inside circle is equal to the above ratio.
	Circle centered on origin.
	Given: -0.5 <= x,y <= 0.5
	A point is inside circle if x**2 + y**2 <= 1/4
	"""
	import matplotlib.pyplot as plt
	from matplotlib import gridspec
	import numpy as np
	plt.clf()
	MAXN = 10000 # Max number of iterations
	n = 1 # Current value of iteration
	n_arr = [] # Saving values of iterations
	pi_arr = [] # Approx values pi for given iteration
	n_circle = 0 # Number of points inside circle
	fig = plt.figure(figsize=(10,15))
	gs = gridspec.GridSpec(2, 1, height_ratios=[2, 1])
	ax = [plt.subplot(gs[0]),plt.subplot(gs[1])]
	# Create a circle just for representation
	circle = plt.Circle((0, 0), radius=0.5, fc='y')
	ax[0].add_patch(circle)
	# Run the simulation
	while n <= MAXN:
	p_color = 'b' # If point is outside circle mark it with blue color
	x = np.random.uniform(–0.5,0.5)
	y = np.random.uniform(–0.5,0.5)
	if (x**2 + y**2) <= 0.25:
	n_circle += 1
	p_color = 'r' # If point is outside circle mark it with blue color
	ax[0].plot(x,y,p_color+'+')
	n_arr.append(n)
	pi_arr.append(n_circle*4.0/n) # Value of pi = 4*points in circle/points overall
	n+= 1
	print n, n_circle, pi_arr[–1], n_arr[–1]
	ax[1].plot(n_arr,pi_arr,"b-",label="monte carlo")
	ax[1].plot(n_arr, np.pi*np.ones(len(n_arr)),"r-", label="actual")
	ax[1].legend()
	ax[1].set_xlabel("Number of iterations")
	ax[1].set_ylabel("Value of pi")
	plt.title("Monte Carlo approximation of pi")

view raw
monte_carlo_circle.py
hosted with ❤ by GitHub

The intriguing thing about this simulation is that using just 10000 points I am getting pretty good value of pi around 3.1392. This approximation however will change with each different run of the code as the randomly generated points change.

I am interested in learning more about such basic applications of monte carlo processes so that the understanding of the more complicated models is easier.

Intel Galileo Gen 2 Board for interfacing internet of things: Getting Started

Intel Galileo Gen 2
Source: https://www-ssl.intel.com/content/www/us/en/do-it-yourself/galileo-maker-quark-board.html

This semester I have signed up for the course INFO 490: DIY with Galileo Development Boards which is a DIY fun course on trying to use the latest Intel Galileo Gen 2 board for variety of playful and useful purposes. I will be trying to document this course in the series of blog posts starting with this post about getting started with the board.

Introduction to Intel Galileo Gen 2:

The intel galileo gen 2 board is a step ahead of the regular embedded systems development boards like Arduino and Raspberry pi in the sense that that is has the full capabilities of Arduino as well as the Intel Quark processor with 256 MB DDR3 RAM. The board thus allows the capabilities of controlling embedded circuits as one would do with an Arduino along with running a linux image which can also be used for controlling the Arduino style GPIO pins on the board. There are tons of resources both on the Galileo website as well as some good tutorials on Sparkfun on how to connect the board and use a demo sketch to use the Arduino side of things as well as details about running a large Linux image on it to do complicated stuff like connecting to the internet. The following are some resources which one can use to get their Galileo board up and running:

• Intel Getting Started – Links to driver, IDE and linux image downloads
• Sparkfun Getting Started – How to get the Arduino Blink example and how to access the console of the board via serial port.
• Intel Galileo community – This is a good community to get most of your problems solved regarding the board and also to interact with the makers and other developers using the board.
• Sparkfun beginner experiments – This is a good start for using the pins on the board using the Arduino interface.
• Raspberry Pi Twitter LED blinker – Good experiment to start using the linux side of the board to interface with the pins.

Using Bigger Linux image via SSH over ethernet:

This is an important and useful thing to learn and configure on the board. If you plan on using more than the Arduino interface and try the Linux interface of the board you will get a lot more flexibility. There are 2 ways to use the console of the board. One is using the Serial port to access the console and another is using SSH connection over ethernet port to the bigger Linux image. Below I will try to explain in detail how to use the SSH over ethernet port which is a much smoother way and can also be use over the network. The usage of serial port can be found on the Sparkfun Getting Started page.

In order to get the SSH connection working you will need the following:

• Ethernet Cable
• Arduino IDE for Galileo configured using any of the getting started
• An SD card with at-least 1-2GB memory with the Bigger Linux Image downloaded from the Intel Download page

Now start the board with the Linux Image on the SD card loaded and upload the first of the following sketches [SSH_IP.ino] to the board:

	# /etc/network/interfaces — configuration file for ifup(8), ifdown(8)
	# The loopback interface
	auto lo
	iface lo inet loopback
	# Wireless interfaces
	iface wlan0 inet dhcp
	wireless_mode managed
	wireless_essid any
	wpa-driver wext
	wpa-conf /etc/wpa_supplicant.conf
	iface atml0 inet dhcp
	# Wired or wireless interfaces
	auto eth0
	iface eth0 inet dhcp
	iface eth1 inet dhcp
	# Ethernet/RNDIS gadget (g_ether)
	# … or on host side, usbnet and random hwaddr
	#iface usb0 inet static
	# address 192.168.7.2
	# netmask 255.255.255.0
	# network 192.168.7.0
	# gateway 192.168.7.1
	iface usb0 inet static
	address 169.254.0.2
	netmask 255.255.0.0
	network 169.254.0.0
	gateway 169.254.0.1
	# Bluetooth networking
	iface bnep0 inet dhcp

view raw
interfaces
hosted with ❤ by GitHub

	void setup() {
	// put your setup code here, to run once:
	system("telnetd -l /bin/sh");
	system("ifconfig eth0 169.254.0.2 netmask 255.255.0.0 up");
	system("sudo service ssh start");
	}
	void loop() {
	// put your main code here, to run repeatedly:
	}

view raw
SSH_IP.ino
hosted with ❤ by GitHub

	void setup() {
	// put your setup code here, to run once:
	system("telnetd -l /bin/sh");
	}
	void loop() {
	// put your main code here, to run repeatedly:
	system("ifconfig eth0 > /dev/ttyGS0");
	}

view raw
SSH_IP_LISTEN.ino
hosted with ❤ by GitHub

This code will configure the IP address of the board as 169.254.0.2 and you should also set the Ethernet IP of your own computer to 169.254.0.1 and use the subnet mask as 255.255.0.0 for both the systems. The code also starts the SSH service on the board which will allow it to accept SSH connections.

Using the second code above [SSH_IP_LISTEN.ino] one can track what is the current IP of the board. This will be useful for the case when you have the IP of the board configured over a DHCP (usually when you connect it to your home router).

A good way to configure the board would be for it to use a static IP when connected to the computer over ethernet and to use DHCP when connected to the router.

For this to work when you SSH into the board using ssh pi@169.254.0.2 you can edit the following file /etc/network/interfaces to the contents of the file 3 [interfaces] shown above. After this you might loose your current SSH connection. Just press the reboot button on the board and wait for it to load again. Once the board has booted up you can either load the 2nd sketch [SSH_IP_LISTEN.ino] and open the Serial Monitor to see the IP of the board and connect to that IP (this IP will usually be different from the static IP you have set up earlier) via SSH. You can also find the IP of your board by logging into your router admin account and checking the IP of the device which is named after the board (default: clanton).

Once you have the SSH running you can try using the various options you get for controlling the board using the command line. I will try to add more info on playing around with the board via linux in future posts.