Enabling Virtio drivers in kernel for running androidx86 on Openstack

Guest operating systems running on virtualised systems needs to cooperate with the underlying hypervisors when using virtualised resources.  Virtio is a set of standard for disk and network virtualisation that is required to be installed on instances that runs on Openstack. The default Androidx86 kernel does not come with these modules installed. Also, you have to edit the source code of Androidx86 OS to detect the virtualised block devices. Otherwise you will see a screen with "Detecting Android-x86... (continuous dots :s)".

This is what you will see in androidx86 debug mode:

If you do not want to compile the source and set it up yourself. I have already created the image for you. Download it from here.

1. First,  to compile the OS you have to initialise the build environment.  Follow instructions here.

2. Download and install the repo client using instructions here.

3. Download Androidx86 source from here.

4. Alter the kernel defconfig files in "kernel/arch/x86/configs/android-x86_defconfig"

and "kernel/arch/x86/configs/android-x86_64_defconfig" by adding following lines.

CONFIG_VIRT_DRIVERS=Y
CONFIG_VIRTIO=y
CONFIG_VIRTIO_PCI=y
CONFIG_VIRTIO_MMIO=m
CONFIG_VIRTIO_BALLOON=m
CONFIG_VIRTIO_BLK=y
CONFIG_VIRTIO_NET=m
CONFIG_VIRTIO_RING=m
CONFIG_VIRTIO_CONSOLE=m
CONFIG_HW_RANDOM_VIRTIO=m

5. To be able to detect virtual block devices, alter the "bootable/newinstaller/initrd/init" file. 

In line 124, Change

for device in ${ROOT:-/dev/sr* /dev/[hs]d[a-z]* /dev/mmcblk*}; do

to

for device in ${ROOT:-/dev/sr* /dev/[hsv]d[a-z]* /dev/mmcblk*}; do

6. Insert following lines to device/generic/x86/init.sh

#Force dhcp on eth0 interface.
netcfg eth0 dhcp
#Start SSH daemon at startup.
start-ssh 

7. Add follwing packages to PRODUCT_PACKAGES in device/generic/x86/packages.mk.

ssh-keygen
sshd_config
start-ssh

8. Then compile.

9. Install androidx86 OS onto a disk storage (dvi, vmdk, qcow2) .

10. To fetch the nova keypair, create fetchsshkeys script with following lines in /data/local/.

#!/system/bin/sh

# Fetch public key using HTTP
cd /data
wget http://169.254.169.254/latest/meta-data/public-keys/0/openssh-key 
cat /data/openssh-key > /data/ssh/authorized_keys

chmod 0600 /data/ssh/authorized_keys
restorecon /data/ssh/authorized_keys

rm /data/openssh-key 

11. Make the script executable.

chmod 755 /data/local/fetchsshkeys

12. Call the script in /etc/init.sh file.

#Fetch ssh keys from openstack metadata service.
/data/local/fetchsshkeys

13. Upload the disk to Glance.

It should now detect /dev/vda1.

14. Create a virtual machine using the uploaded image. Your VM should get a private ip. (Make sure to specify a keypair)

15. You should be able to ssh into the androidx86 VM now.

16. To get get GUI, add "nomodeset" to grub.

Thanks to everyone helped me on this issue from Androidx86 mailing list.

Openstack Grizzly slow API and timeouts

I have come across this issue a while ago on our Openstack testbed that as the amount of API calls increase, the delay when you do API calls increased.  Then I discovered that Openstack does not clean the expired tokens automatically when you are using a SQL backend.

https://bugs.launchpad.net/ubuntu/+source/keystone/+bug/1032633

To delete the expired tokens I have created a script so that I can automate this, by creating a cron job.

You can find the script here on my github. Download the script and change the password to your mysql password.

Then create a cron job to periodically execute this script.

0 0 * * * /path/to/keystone_flush 2>&1 >> /path/to/log/keystone_flush.log

Above cron job will run ones everyday. You may change the timing of the job as you desire.

However this problem is solved in Openstack Havana release.

https://blueprints.launchpad.net/keystone/+spec/keystone-manage-token-flush

Openstack Folsom cant ping instance/VM

The most annoying bug that I came across when setting up Openstack Folsom was that, after creating the a virtual machine it was not able to reach the metadata service and I was not able to reach the virtual machine.

After days of debugging, I found out that it was not only me that was having the same issue. Finally it was reported as a known bug to the openstack developers.

https://bugs.launchpad.net/neutron/+bug/1091605

The problem seems to be that OpenvSwitch devices have been brought back up after after rebooting.  Hence before I upgraded to Grizzily from Folsom, I created the following script and had it running at the boot-up in the network node.

 #!/bin/sh  
   
 #  
 # This script fixes the following bug.  
 # Has to run after reboot.  
 # Bug: https://bugs.launchpad.net/quantum/+bug/1091605  
 #  
 #  
 # Chathura S. Magurawalage  
   
 ((  
 #Check for root permissions  
 # Make sure only root can run our script  
 if [ "$(id -u)" != "0" ]; then  
   echo "This script must be run as root"  
   exit 1  
 fi  
   
 BRIDGE=${BRIDGE:-br0}  
   
 while getopts b:hv option  
 do   
   case "${option}"  
   in  
     b) BRIDGE=${OPTARG};;  
      v) set -x;;  
     h) cat <<EOF   
 Usage: $0 [-b bridge_name]  
   
 Add -v for verbose mode, -h to display this message.  
 EOF  
 exit 0  
 ;;  
      \?) echo "Use -h for help"  
        exit 1;;  
   esac  
 done  
   
 ovs-vsctl del-br br-int  
 ovs-vsctl del-br br-ex  
 ovs-vsctl add-br br-int  
 ovs-vsctl add-br br-ex  
 ovs-vsctl add-port br-ex $BRIDGE  
 ip link set up br-ex  
   
 service quantum-plugin-openvswitch-agent restart  
 service quantum-dhcp-agent restart  
 service quantum-l3-agent restart  
   
 )) 2>&1 | tee $0.log  
   

Ubuntu Kernel panic when creating Openstack instance

Currently I am using Openstack Folsom, nova version  2013.1.4, Installed on Ubuntu 12.04.  I upgraded my compute and network nodes's kernel to  "Linux 3.2.0-58-generic". Since then, when I try to create an instance on a the compute nodes or when I restart the network node, the systems goes to Kernel panic. 

The only fix for this issue I found, is to downgrade the kernel from "Linux 3.2.0-58-generic" to "Linux 3.2.0-55-generic" on all compute nodes and network node to keep things consistent.  The controller nodes does not seem to be affected by this kernel bug.

Also make sure to install the header if the corresponding kernel, if you are installing the kernel. Or if you have already got "Linux 3.2.0-55-generic" kernel installed, then just change it to the default kernel in grub records. 

First find out the index of the kernel in the grub records.

grep menuentry /boot/grub2/grub.cfg

Then the fist answer of the following question should help when editing the grub config.

http://askubuntu.com/questions/216398/set-older-kernel-as-default-grub-entry

Openstack Grizzily Error "FATAL: Module openvswitch_mod not found" when start openvswitch or at startup

I came across this error on Openstack Grizzily installed on Ubuntu 12.04 LTS, when I upgrade or degrade the linux kernel. So its very likely that you will get this error after running a "dist-upgrade" on your system.

The cause of this problem seem to be that when a different kernel is used, the openvswitch modules have not been built  with it.

Error:
FATAL: Module openvswitch_mod not found.
 * Inserting openvswitch module
 * not removing bridge module because bridges exist (virbr0)
invoke-rc.d: initscript openvswitch-switch, action "load-kmod" failed.

Solution:

Copy the following script to a file. Then save.

 #!/bin/bash  
   
 sudo apt-get install linux-headers-$(uname -r)  
   
 depmod  
 modprobe openvswitch  
   
 sudo apt-get remove openvswitch-switch openvswitch-datapath-dkms quantum-plugin-openvswitch-agent  
   
 sudo apt-get install openvswitch-switch openvswitch-datapath-dkms quantum-plugin-openvswitch-agent  
   
   

Then make it make the file executable.

Go to the file directory then execute following command in terminal:

sudo chmod +x filename

Finally execute the file.

./filename 

Openstack Folsom Error "FATAL: Module openvswitch_mod not found" when start openvswitch or at startup

I came across this error on Openstack Folsom installed on Ubuntu 12.04 LTS, when I upgrade or degrade the linux kernel. So its very likely that you will get this error after running a "dist-upgrade" on your system.

The cause of this problem seem to be that when a different kernel is used, the openvswitch modules have not been built  with it.

Error:

FATAL: Module openvswitch_mod not found.
 * Inserting openvswitch module
 * not removing bridge module because bridges exist (virbr0)
invoke-rc.d: initscript openvswitch-switch, action "load-kmod" failed.

Solution:

Copy the following script to a file. Then save.

 #!/bin/bash  
   
 sudo apt-get install linux-headers-$(uname -r)  
   
 sudo apt-get remove quantum-plugin-openvswitch openvswitch-switch quantum-plugin-openvswitch-agent openvswitch-datapath-dkms openvswitch-common quantum-common python-quantum  
   
 apt-get install openvswitch-switch  
 mkdir -p /etc/quantum/  
 apt-get install quantum-plugin-openvswitch-agent quantum-dhcp-agent quantum-l3-agent quantum-server  
   
 depmod  
 modprobe openvswitch  
   

Then make it make the file executable.

Go to the file directory then execute following command in terminal:

sudo chmod +x filename

Finally execute the file.

./filename 

Lunux/Ubuntu not booting up after the kernel update?

After I have upgraded to 2.6.38-9-generic kernel I got the following message and I have tried many thing but this is how I have solved the problem.

Gave up waiting for root device. Common problems:
-Boot args (cat /proc/cmdline)
-Check rootdelay= (did the system wait long enough?)
-Check root= (did the system wait for the right device?)
-Missing modules (cat /proc/modules; ls /dev)
ALERT! /dev/disk/by-uuid/310147a9-f21c-4e7c-aeb0-18f31073da58 does not exist. Dropping to a shell!

BusyBox v1.10.2 (Ubuntu 1:1.10.2-2ubuntu7) built-in shell (ash)
Enter 'help' for a list of built-in commands.

(initramfs)

Boot your computer and when the grub loader appears, select the linux version you use (to the rescue mode) and press 'e' to edit it.

Then remove the the UUID= part of the line from root=UUID=

Replace it with your current partition, so in my case it looks like

root=/dev/sda5

press +x to boot up with the changed settings

Then after you have logged in install fglrx module.

sudo apt-get install fglrx

Then restart.

Now this time select your linux version but this time not the rescue mode.

Then replace the UUID how you did earlier.

Done.

To permanently apply the settings to grub, you will have to edit the grub.cfg file

sudo vi /boot/grub/grub.cfg

Intelligent Agents in a simulated environment

The following video and the description is about a Agent simulation that I have designed and implemented. I would like to give my gratitude to the "Intelligent Agents" module director 

Dr. Sam Steel at the University Of Essex for his supervision throughout the module.

The simulation shows an "Agent-world" represented by a grid of squares. Agents survive in this world by eating plants. There are four types of plants that occur in various quantities. Agents are able to consume a combination of plants and gain more energy. The energy levels decreases as they live and as they roam to other squares in the gird. Creatures can carry a limited amount of plants so then can eat when it is needed. If the creatures can create two combinations at once then the most energetic combinations of plants will be eaten first.

In the “Agent world” there could be maximum of two types creatures in the world.

The agent types:

SimpleCreature: Less intelligent creature.

IntelCreature: Most intelligent creature.

SimpleCreature:

A simple creature will be born in the agent world with a limited amount of energy. The creatures move around the world in random directions in a consecutive manner, for random amounts of times, with an intention to feed to survive. A simple creature can carry a limited amount of plants. This creature eats plants when it's energy goes down less than it's energy limit. (When it feels hungry) The creature is able to consume compounds(CP)of plants (P) of varied types (t) that they carry, and gain more energy than it would if it eats them individually. Such that:

CP = {Pt₁, Pt₂, Pt₃, Pt₄}

Creatures can only see the plants in the same square as it is. When the world has been resized, eventually the creatures will follow back in to the world. Each individual creature has got a unique ID. This phenomenon has been recorded on the video.

The simulated environment in which the creatures act:
The “Agent world” is a grid of squares. The plants and the creatures will be in this grid.

IntelCreature: 

Only an IntelCreature can look around. These creatures can only see only one square around ahead, as well as the square it is interpolated. They can also see the the plants as well as the creatures in its range of vision.

The creature can remember the creatures and the plants in the squares that it can see (This memory is limited). Although the vision is perfectly accurate only within its local square. Just after it has been born, IntelCreature will be in an Observation mode, so that it will observe the world by capriciously observing the world until it finds more than one plant. Then it will start going after the plants intentionally that it sees around. A creature may decide to go after plants that are more therapeutic than the others. Such plants may be the plants that could complete a combination.

If two creatures meet together in one square they start talking to each other, asking about the plants that the other creature carry. Then the first creature will propose to the other creature for a shared combination, with the amount that it would like to share. Such that:

Assume:
combo1 = P1 + P2 + P2
Ag₁:Plants = {P1, P3, P3, P1}
Ag₂:Plants = {P2, P2, P3}

Then:

Ag1: Ag2.requestCombo(<share_amount>, <plant>)

If the other creature agrees then they will consume the plants and share the energy amount. When they share the plant combinations, they will get the double the amount of the normal combination, hence it is more desirable for the creatures to share. The creatures can decide to be partners with another one, and go for other plants that one of the creatures have seen. Furthermore when the creatures are in the same square, they may request other creatures to be parters, a creature will request to be a partner of another creature, if the plant the second creature is going after, interests the first creature. If the second creature agrees to be partners, then it will make a leader and partner relationship between each other. The first creature that made the request will be the partner and the other will be the leader.  After the leader gets the plant, the energy amount will be shared. Finally the partnership will end and the two creatures will go their own ways. In the situations where the plants that are subject to the partners engrossment have been taken by some other creature, the partnership will immediately end. When the partnership has been made, the colour of the red circle, will change to a light yellow colour if it is a partner, and the colour of the leader's circle will change in to dark yellow. This is demonstrated in the figure blow.

A creature can make partnership with another one, only if the both of the creatures are neither a partner or a leader. If the creatures do not carry any plants and it can not remember any plants, it goes back to the Observation mode to collect information about the world(Grid). If the creature can not carry any more plants, it will stop travelling to save the amount of energy that it will loose by going to other squares.

What are plants?:

• There are four types of plants in the world.
• Green plants, Red plants, Yellow plants, and Magenta plants.
• The plants are represented in small squares with the colour of their names.
• Each plant holds different energy levels.
• A plant set consists of a set of random types of plants.
• The plant sets appears in the “Agent world” in arbitrary places.
• The amount of the plants that will be put in to the grid is variable. 
• In the world there will be a constant amount of plants at a time.
• The size of the “Agent world” is variable.
• The number of types of plants are in the world are variable.
• The energy of creatures will be consumed until it dies. The energy consumption speed is variable.
• If the creature moves to a different square, more energy will be consumed than if it stays in the same square.

The log output of the software which show

• what a particular simple creature does
• what a particular creature that looks around does
• the interaction of a two creatures who meet 

The above description about the "AgentWorld" is just a brief introduction about the project. Accordingly there are many other theoretical and technical details that I have not introduced to you in this article. Although if you are interested, I am happy to supply you with further details on this project.

Content Based Image Retrieval using cross correlation co-efficiency

Have you ever searched google for images? I am sure everyone who is reading this article has done this before more than once. Wouldn't it be nice if you could search images for the similar images, for a given image? So for an example, if you want to search for an image of a particular object, by supplying a probe image (a image which to find similar images of) to the search engine, you can search for images that are similar to the given image.

Content-based image retrieval (CBIR) or Query by image content (QBIC), is an active area in Computer Vision research. Comparing images using cross-correlation efficiency was similar to the state of the art for CBIR in the early 1990s. Comparing the colours in the probe image and other test images, is one conservative way of doing this. Histogram is a good way of discriminating the colours in an image.

One way of doing this is explained below. There are many ways of doing this although this is not necessarily the right or the best way of doing it.

1. Split the R,G,B channels of the probe image
2. Split the R,G,B channels of the "test image" that is subject to comparison.
3. Calculate the histograms of the R,G,B channels of each images.
4. Calculate the cross correlation of the R, G, B channels of the probe image and the "test image" histograms. Such that, 

    r1 = correlation(img1.R,img2.R);
    r2 = correlation(img1.G,img2.G);
    r3 = correlation(img1.B,img2.B);

     5. Finally multiply the results all together ( r1 x r2 x r3 )

The result (r) of a cross correlation calculation lies between -1 and +1. If the value is +1 in average the two images are identical. If it is -1 then the images are identical but the image looks like a photographic negative version of the probe image. If r=0 then the images are on average very unlike each other.

People who are interested in Computer Vision are likely to be familiar with the OpenCV library. The following code snippet (incomplete) is an example for calculating the cross correlation co-efficiency using the OpenCV library in C.

  //split img1 in to RGB channels and store them in seperate images

  cvSplit(img1, img1R, img1G, img1B, NULL);

 

  //calculate histograms of the the image

  CvHistogram *h1R = calHistogram(img1R, bins, ranges);

  CvHistogram *h1G = calHistogram(img1G, bins, ranges);

  CvHistogram *h1B = calHistogram(img1B, bins, ranges);



    //split the image two in to RGB channels and store them

    //in seperate images

    cvSplit(img2, img2B, img2G, img2R,0);

   

    //calculate histograms of the RGB channel images of the "img2"

    CvHistogram *h2R = calHistogram(img2R, bins, ranges);

    CvHistogram *h2G = calHistogram(img2G, bins, ranges);

    CvHistogram *h2B = calHistogram(img2B, bins, ranges);

      

    //Calculate the correlation coefficient of RGB images, of img1 and img2

    rR = cvCompareHist(h1R,h2R,CV_COMP_CORREL);

    rG = cvCompareHist(h1G,h2G,CV_COMP_CORREL);

    rB = cvCompareHist(h1B,h2B,CV_COMP_CORREL);

    

    //Multiply the results together

    tempR= rR*rG*rB;

The performance:

Some performance evaluation has been done on this program using a test harness called 'Fact'.

Error rates calculated from mycbir 



tests   TP         TN       FP        FN   accuracy  recall precision specificity       class 



8        0         0         8         0     0.00     0.00    0.00        0.00   box 



8        0         0         8         0     0.00     0.00    0.00        0.00   cockeral 



8        6         0         2         0     0.75     1.00    0.75        0.00   dog 



9        7         0         2         0     0.78     1.00    0.78        0.00   horus 



8        7         0         1         0     0.88     1.00    0.88        0.00  ornament 



10        8         0        2         0     0.80     1.00    0.80        0.00   penguin 



8        1         0         7         0     0.12     1.00    0.12        0.00   reindeer 



9        2         0         7         0     0.22     1.00    0.22        0.00   tortoise 



68       31        0        37         0     0.46     1.00    0.46        0.00          overall

TP = True Positives
TN= True Negatives
FP = False Positives
FN= False Negatives


In the matrix above it shows the amount of test cases for the each picture in the “test” column, True Positives, True Negatives, False Positives and False Negatives in the next four columns.  Then the application calculates the accuracy, recall, precision and specificity as follows, and displays in the next four columns. The last “class” column show the type of picture that it has carried out the test on,

Precision is the number of relevant pictures that are retrieved, divided by the total number of of pictures retrieved. Recall is the number of relevant pictures retrieved divided by the total existing relevant pictures. The specificity and the accuracy is calculated as below

                                      TP                                                                        TN                                    

sensitivity = recall = ------------------                                       specificity =     -----------------

                                TP + TN                                                                   FP + TN

 As you can see above, the application has failed to find the True Positives for the “box” and “cockeral” pictures. So in contrast it has found False Positives in all cases. The best case is the tests on the ornament pictures as it has found 7/8 TPs and 1/8 FPs. Overall this algorithm has done 68 test cases and have found 31 True Positives and 37 False Positives, but no True Negatives or False Negatives.

The matrix below, is the class confusion matrix.

Confusion matrix calculated from mycbir 



expected 



actual    box    cockeral         dog    horus   ornament  penguin reindeer tortoise 



box        0        0              0        0        0        0        0        1 



dog        0        0              6        0        0        0        0        0 



horus      0        0              1        7        0        0        0        4 



ornament   1        1              0        0        7        2        7        0 



penguin    1        0              0        0        1        8        0        0 



reindeer   5        7              0        1        0        0        1        2 



tortoise   1        0              1        1        0        0        0        2

The columns and rows indicates that, the classes in the rows titles were obtained when the classes in the column title should have been found (Rows as obtained results, columns as expected result). The numbers indicates the number of times it has been obtained. So as an example, as the worst case: in the “box” column,

ornament =1, penguin = 1, reindeer = 5, tortoise = 1 has been obtained when “box” pictures should have been found. But it has not obtained any “box” pictures as the value of the “box” raw is 0.

As I have mentioned earlier the best results has been obtained on the tests with the “ornament” picture. It has obtained the correct “ornament” pictures 7 times and a penguin picture just one time.

Conclusion:

Cross correlation is not the best Content based Image Retrieval technique to date, though it helps us to understand how the histograms of two images can vary or can be complementary. But it is no doubt that analysing just the colours in images are a good way of characterising them though it is hard to compare two images using just this factor.

Install and configure trac on Linux/Ubuntu 11.10

In this blog post I am going to explain about how to install 'trac' on linux. 'trac' is a nice project management tool which is written in python, that can be used to track your projects. And it is open-source :D (Win)

I am using 'trac' with 'git' revision control system, for my project. In my next blog post I will be explain how to use 'trac' along with 'git', using the 'GitPlugin'

 Ok, now if you follow the following instructions hopefully you will have a working 'trac' site.

Install 'trac'
   
Make sure you have the latest version of python installed, otherwise apt will install an older version of 'trac'

      sudo apt-get install python python-babel trac

Yes, apt may ask you to install few dependencies including apache2. So please install all of them.

'trac' uses SQLite as its default data store. So if you have not already got it installed. Then do the following

     sudo apt-get install sqlite3

It is possible to get 'trac' working with other database systems. If you are interested, click on the following URL.
http://trac.edgewall.org/

Upgrade python packages
   
There are two ways you can do this

With easy_install
 

 easy_install Babel

 easy_install Trac

or

With 'pip'
 

 pip install --upgrade Babel

 pip install --upgrade Trac

If you have not got pip installed, install pip as follows. Please skip this step if you were successfully able to do the previous step.

 sudo apt-get install python-pip

   

Now you have successfully installed all the software that you need to run 'trac'.
   
Configuration

  Change the directory locations as desired.

 Create a directory for 'trac'
 Then change it's access permissions.

sudo mkdir -p /var/local/trac && chown www-data: /var/local/trac

   
Initialise the 'trac' environment
 

 sudo trac-admin /var/local/trac initenv

   

Now create the configuration file using your favourite text editor. I use emacs.
 

 sudo emacs /etc/apache2/sites-available/trac

   

Paste the following code in to it.

       
<VirtualHost *:80>

 ServerName trac.local

        <Location />

           SetHandler mod_python

           PythonInterpreter main_interpreter

           PythonHandler trac.web.modpython_frontend

           PythonOption TracEnv /var/local/trac

           PythonOption TracEnvParentDir /var/local/trac

           PythonOption TracUriRoot /

           PythonOption TracEnv /var/local/trac

            # PythonOption TracEnvIndexTemplate /var/local/trac/templates/index-template.html

           PythonOption TracLocale en_US.UTF8

           PythonOption PYTHON_EGG_CACHE /tmp

           Order allow,deny

           Allow from all

        </Location>

        <Location /login>

          AuthType Basic

          AuthName "myproject"

          AuthUserFile /var/local/trac/.htpasswd

          Require valid-user

        </Location>

      </VirtualHost>

 Now you should have a working 'trac' instance. So check if it works.

      sudo tracd -p 8080 /var/local/trac

[-p] flag specifies the port that this particular 'trac' instance belongs to.
   
Note: the flag [-p] is same as [-port]

And then go to:

      http://localhost:8080/

Now you should see the 'trac' instance running.

Adding Authentication
   
Basic Authorisation

In this case we are going to authorise the 'trac' site with a .htpasswd file.

You have to have 'fcrypt' package installed to decode '.htpasswd'

Creating the '.htpasswd'

 sudo htpasswd -c /var/local/trac/.htpasswd admin

To add more users:

 sudo htpasswd -c /var/local/trac/.htpasswd admin

   
'htpasswd' creates the flat-file with the username and password that you are given.
[-c]  =  create the password file in the given path.

Start tracd:

 sudo -p 8080 --basic-auth="projectdirectory,path/to/the/.htpasswd,mname" /path/to/the/environment/directory

 In my case, it is

 sudo -p 8080 --basic-auth="trac,/var/local/trac/.htpasswd, admin" path/to/the/environment/directory

   

 Digest Authentication
   
 'htdigest' will be used to create the digest file.

 sudo htdigest -c /var/local/trac/.htdigest admin admin

  Now start 'trac' with 'tracd':

 sudo -p 8080 --auth="projectdirectory,path/to/the/.htpasswd, admin" /path/to/the/environment/directory

  So it will be,

 sudo -p 8080 --auth="trac,/var/local/trac/.htpasswd, admin" /var/local/trac

  If you have just one project in 'trac', then use the [-s] flag with the 'tracd', so it will skip the environment list when it starts.
      eg: sudo tracd -s ...........

There we go, we are done :D

Trouble Shooting Errors


If you get following error. It is most likely because of permission issues. So make sure you have given root access when you start your 'trac' instance.

Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/trac/web/api.py", line 440, in send_error
    data, 'text/html')
  File "/usr/lib/python2.7/dist-packages/trac/web/chrome.py", line 827, in render_template
    message = req.session.pop('chrome.%s.%d' % (type_, i))
  File "/usr/lib/python2.7/dist-packages/trac/web/api.py", line 216, in __getattr__
    value = self.callbacks[name](self)
  File "/usr/lib/python2.7/dist-packages/trac/web/main.py", line 300, in _get_session
    return Session(self.env, req)
  File "/usr/lib/python2.7/dist-packages/trac/web/session.py", line 198, in __init__
    self.get_session(sid)
  File "/usr/lib/python2.7/dist-packages/trac/web/session.py", line 219, in get_session
    super(Session, self).get_session(sid, authenticated)
  File "/usr/lib/python2.7/dist-packages/trac/web/session.py", line 61, in get_session
    db = self.env.get_db_cnx()
  File "/usr/lib/python2.7/dist-packages/trac/env.py", line 328, in get_db_cnx
    return get_read_db(self)
  File "/usr/lib/python2.7/dist-packages/trac/db/api.py", line 90, in get_read_db
    return _transaction_local.db or DatabaseManager(env).get_connection()
  File "/usr/lib/python2.7/dist-packages/trac/db/api.py", line 152, in get_connection
    return self._cnx_pool.get_cnx(self.timeout or None)
  File "/usr/lib/python2.7/dist-packages/trac/db/pool.py", line 226, in get_cnx
    return _backend.get_cnx(self._connector, self._kwargs, timeout)
  File "/usr/lib/python2.7/dist-packages/trac/db/pool.py", line 146, in get_cnx
    raise TimeoutError(errmsg)
TimeoutError: Unable to get database connection within 0 seconds. (TracError(<babel.support.LazyProxy object at 0x22e47d0>,))

If you need more information, please go to http://trac.edgewall.org.

Set up Alpine to work with Gmail

To set up Alpine with Gmail, You have you to have Alpine installed on your computer. To set up Gmail, click here to go to my previous blog post which explains how to install it.

If you have got Alpine installed then, next

To receive emails from your gmail account:

Go to the Main Menu >> Setup >> collectionLists >> Add.


Nickname  : [Your Nickname]
Server        : imap.gmail.com/ssl/user=username@gmail.com
Path           :
View         :

Save your settings

You can add as many accounts you want if you press E and exit, then do the same.

Then you need to change few settings on it.

On Main menu > Set up > Config, find the "Advanced user preferences", Then mark "Save will not delete"

Then scroll down and find Pruning Rule and select "don't rename, don't delete"

To send emails using your gmail account :

Go to Main menu >> Rules and then Add a new rule.

    Add a Nickname
    In the Current Folder Type section, Select "Specific"
    Then select "Folder List" and press "T" and select the specific account folder(Inbox)
    Scroll down and find the section where it says "Action Begin here"
    Insert (Add) your email address to "Set Form" field
    Then to the "use SMTP" server section Add "smtp.gmail.com/tls/user="   (Without the quotes)
    Next Scroll down and find the "Compose Use" value and then select "With Confirmation"
    Then Exit
    You need to Add  a new rule per each account

Now you are ready to go!

Read/Receive and Send emails on the linux command line - Alpine/pine

I like to use my command line to do most of the work rather than using GUIs. So I wanted to be able read and send emails using the linux command line. After done much research I have found this free application called "Alpine" developed at the University Of Washington, which the earlier versions of it was called "Pine".

To download you can go to the official website by clicking the the URL below
http://www.washington.edu/alpine/

or if you are a Debian/Ubuntu user you can simply type in the command below

sudo apt-get install alpine

Currently I am using Alpine with my gmail account. I may be do a post about how to configure gmail on Alpine

Enjoy

Secure your Gmail/Google Account

      Are you confidant that your email account is secure anymore? You can never make anything 100% secure but, there are things you can do to make it more secure than it is.

     Gmail has introduced 2-step verification which protects your gmail account little bit more. I recommend that everyone who read this blog, should enable the feature.

To enable: Go to your Google account page and then click on the
                  "Using 2-step verification" And then follow the instructions

Make sure you read the instructions carefully and print out or keep the verification codes safely, which you will need to log in to your account when you haven't got your phone to get a verification code. Never loose them.

The other thing you could do to protect your Gmail account is that to always use HTTPS. To enable this feature.

Log in to your gmail account
Click on the Options symbol
Click on the General tab
Then click on the radio button which says "always use https", under "Browser Connection"
Save Changes

Surf Save!

Start Dradis in BackTrack 5

Back Track 5 comes with Dradis pre loaded already
But you might not be familiar with the way you start Dradis in BackTrack 5

To start Dradis:

Go to the dradis/server directory 

root@bt:~# cd /pentest/misc/dradis/server

Then run start.sh script in the dradis directory 

root@bt:/pentest/misc/dradis/server# /pentest/misc/dradis/start.sh

No Active Driver when you type in db_driver or db_connect?

Do you get the following error message when you type in db_driver on Metasploit console on BlackTrack5?

[*] No Active Driver
[*]        Available:

[*]     DB Support: Enable the mysql driver with the following command:
[*]                 $ gem install mysql
[*]     This gem requires mysqlclient headers, which can be installed on Ubuntu with:
[*]                 $ sudo apt-get install libmysqlclient-dev

[*]     DB Support: Enable the postgresql driver with the following command:
[*]                   * This requires libpq-dev and a build environment
[*]                 $ gem install postgres
[*]                 $ gem install pg # is an alternative that may work

msf >

And you have tried installing mysql driver but didn't solve the problem ?

This happens because when you start msfconsole by /pentest/exploits/framework3/msfconsole, it does not set the environment variables properly

So all you have to do is, start msfconsole using BT5 menu or by using /usr/local/bin/msfconsole

Problem With psexec (Solution)

I do my pentests on a Windows XP sp3 box and I had this problem, that I could not get psexec to work as I kept getting this error message.

[*] Started reverse handler on :4444
[*] Connecting to the server...
[*] Authenticating to :445|WORKGRO[*] Started reverse handler on 192.168.56.3:4444
[*] Connecting to the server...
[*] Authenticating to :445|WORKGROUP as user ''...

[-] FAILED! The remote host has only provided us with Guest privileges. Please make sure that the correct username and password have been provided. Windows XP systems that are not part of a domain will only provide Guest privileges to network logins by default.

[*] Exploit completed, but no session was created.UP as user ''...

[-] FAILED! The remote host has only provided us with Guest privileges. Please make sure that the correct username and password have been provided. Windows XP systems that are not part of a domain will only provide Guest privileges to network logins by default.

[*] Exploit completed, but no session was created.

That was because if your Windows XP Professional based computer is not in a domain, by default all users trying to log in to it over the network, are forced to use the Gust Account. To find out more information go to http://www.windowsnetworking.com/articles_tutorials/wxpsimsh.html

So the solution is to

Go to HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Lsa
And change the ForceGuest value to 0

Done!

Text Encryption Application

Under the Computer Security module we did one last assignment at the end of the term, the assignment was to implement the substitute cypher and demonstrate how it can be broken by using Frequency Analysis. So I have written a code to encrypt some text using a key, which the user inputs to the application (Which the application assumes that the key is unique), and to Break the encryption using frequency analysis, using Java. It was required to leave the spaces and non alphabetic characters, and maintain the cases in the cipher text (which makes the cypher text weaker).

I have added another extra feature for the cipher breaking code called The "Advanced Frequency Analysis". Which means that, In frequency analysis it just analyses the frequencies of single characters. But if you turn on this feature if will analyse two and three character combinations as well as single characters, which makes the code breaker more strong.

You will probably wonder that why would I want to add extra code to make the encryption weak. As an example this program passes spaces, non alphabetical characters through to the cipher text. It is because that they are requirements of the assignments so students can write about it on the report and give explanations about how to make it more stronger.

Find the screen shots and examples of the program below.

Encryption

Plain Text:

Data centres use vast amounts of electricity to run their computer equipment and also to keep it cool.

Environmental group Greenpeace has estimated that their total global energy use will have reached 2 trillion kw/h by 2020.


Key:

qazxswedcvfrtgbnhyujmkiolp

Cipher Text:

Hjtj koetfor gro bjrt jwdgetr dl osoktfakaty td fge tqoaf kdwcgtof ovgacwoet jei jsrd td xooc at kdds.

Yebafdewoetjs pfdgc Jfooecojko qjr ortawjtoi tqjt tqoaf tdtjs psdmjs oeofpy gro nass qjbo fojkqoi 2 tfassade xn/q my 2020.


Decryption

Advanced Frequency Analysis = off

Decrypted plain text:


Kata lestier dre wart amodstr oq eheltinlntg to ids tueni lomc1dtei ezdnc1mest asf ahro to veec1 nt looh. Jswniosmestah yiodc1 Xieesc1eale uar ertnmatef tuat tueni totah yhopah eseiyg dre bnhh uawe iealuef 2 tinhhnos vb/u pg 2020.


Advanced Frequency Analysis = on

Decrypted plain text:

Kata lestier the wart amodstr in eheltinlntg of for tueni lomc1dtei ezdnc1mest and ahro of veec1 to looh. Jswniosmestah yiodc1 Xieesc1eale his ertnmatef tuat tueni totah yhopah eseiyg the bnhh uawe iealuef 2 tinhhnos vb/u it 2020.



My Example is just a simple demonstration. As you can see its hard to analyse and decrypt just by using an application/ Algorithm, without having any human help. But This algorithm works better as the plain text gets larger, as it gets more data to analyse so the accuracy of the analysis gets better.

My Tetris

'Tetris' is one of the most popular games of all time. For the last project of Application Programming module at the university, I got to develop this small Tetris game applet. As you can see its very basic and simple as I added more than requirements of the assignment, to the application itself. It would have had more functionalities but I moved on from it to the next assignment so I can finish my assignments and get it everything done way before the deadline. I will add more features to it if I get much time in the future, even though I tried ones and then I got busy with something else and then moved on.

To control the blocks You can use either keyboard or mouse or both.

Controls:

Mouse Controls:

Right-Mouse Button- Move right
Left-Mouse Button - Move Left

Key Board Controls:

Right Arrow key- Move Right
Left Arrow key - Move Left
'R' key - Rotate

If I develop it more further I will definitely post it on here. Like I have said it earlier its a very simple version of Tetris. But it plays according do all Tetris rules.

John Conway’s Game Of Life

  

    Game of life  is devised John Conway is a British mathematician John Horton Conway in 1970. As a part of my last Java assignment I re designed and added more components to the game under the Java lecturer Professor Simon M. Lucas’s guidance. I was really excited to make this application just after seeing Simon’s example of the Game Of Life.

 

Rules:

The universe of the Game of Life is an infinite two-dimensional orthogonal grid of square cells, each of which is in one of two possible states, live or dead. Every cell interacts with its eight neighbours, which are the cells that are directly horizontally, vertically, or diagonally adjacent. At each step in time, the following transitions occur:

 

1. Any live cell with fewer than two live neighbours dies, as if caused by under-population.
2. Any live cell with more than three live neighbours dies, as if by overcrowding.
3. Any live cell with two or three live neighbours lives on to the next generation.
4. Any dead cell with exactly three live neighbours becomes a live cell, as if by reproduction.

The initial pattern constitutes the seed of the system. The first generation is created by applying the above rules simultaneously to every cell in the seed—births and deaths happen simultaneously, and the discrete moment at which this happens is sometimes called a tick (in other words, each generation is a pure function of the one before). The rules continue to be applied repeatedly to create further generations  

-Wikipedia-

 

 

 

            

 

You can create your own rules by changing the bits on the bit editor in the right hand side. You can stop and Resume your game anytime by just pressing the Start/Stop button. By changing bits you can create really cool patterns on the game window. As an example the following picture shows one of the rules I have made

 

            

I won’t talk about the Java code and the logics of this program but If you want to know or if you want to play this game,  just email me or leave a comment. I’ll send you more information

 

Thank you for visiting My blog

Scribble

 

   Scribble is a painting pad which enables you to draw really cool patterns on it. I made this peace of application as a part of the java assignment under the Java lecturer Professor Simon M. Lucas’s guidance. It’s been a great experience to make this application.  It was a great experience to make Scribble. I was quite excited to see the end product after seeing completed application in the Lectures. I have completed the whole assignment successfully before the dead line and I am glad that I have got quite high marks for it and I have got full marks for this part of the assignment . I will soon post the other past of the assignment and it will be my next blog post.I have posted some random things I drew on it so you can get an Idea about what it looks like. Hope you will like it.

 

 

 

 

                      

 

 

 

                       

 

 

                      

 

 

 

I wont post anything about the Java Codes of this application but if you want to know about it or if you want to try this application by yourself,  just email me or leave a comment.

 

Thank you for visiting my blog

 

-Chathura Mazz-