Openstack Grizzly slow API and timeouts

I have come across this issue a while ago on our Openstack testbed that as the amount of API calls increase, the delay when you do API calls increased.  Then I discovered that Openstack does not clean the expired tokens automatically when you are using a SQL backend.

https://bugs.launchpad.net/ubuntu/+source/keystone/+bug/1032633

To delete the expired tokens I have created a script so that I can automate this, by creating a cron job.

You can find the script here on my github. Download the script and change the password to your mysql password.

Then create a cron job to periodically execute this script.

0 0 * * * /path/to/keystone_flush 2>&1 >> /path/to/log/keystone_flush.log

Above cron job will run ones everyday. You may change the timing of the job as you desire.

However this problem is solved in Openstack Havana release.

https://blueprints.launchpad.net/keystone/+spec/keystone-manage-token-flush

Openstack Folsom cant ping instance/VM

The most annoying bug that I came across when setting up Openstack Folsom was that, after creating the a virtual machine it was not able to reach the metadata service and I was not able to reach the virtual machine.

After days of debugging, I found out that it was not only me that was having the same issue. Finally it was reported as a known bug to the openstack developers.

https://bugs.launchpad.net/neutron/+bug/1091605

The problem seems to be that OpenvSwitch devices have been brought back up after after rebooting.  Hence before I upgraded to Grizzily from Folsom, I created the following script and had it running at the boot-up in the network node.

 #!/bin/sh  
   
 #  
 # This script fixes the following bug.  
 # Has to run after reboot.  
 # Bug: https://bugs.launchpad.net/quantum/+bug/1091605  
 #  
 #  
 # Chathura S. Magurawalage  
   
 ((  
 #Check for root permissions  
 # Make sure only root can run our script  
 if [ "$(id -u)" != "0" ]; then  
   echo "This script must be run as root"  
   exit 1  
 fi  
   
 BRIDGE=${BRIDGE:-br0}  
   
 while getopts b:hv option  
 do   
   case "${option}"  
   in  
     b) BRIDGE=${OPTARG};;  
      v) set -x;;  
     h) cat <<EOF   
 Usage: $0 [-b bridge_name]  
   
 Add -v for verbose mode, -h to display this message.  
 EOF  
 exit 0  
 ;;  
      \?) echo "Use -h for help"  
        exit 1;;  
   esac  
 done  
   
 ovs-vsctl del-br br-int  
 ovs-vsctl del-br br-ex  
 ovs-vsctl add-br br-int  
 ovs-vsctl add-br br-ex  
 ovs-vsctl add-port br-ex $BRIDGE  
 ip link set up br-ex  
   
 service quantum-plugin-openvswitch-agent restart  
 service quantum-dhcp-agent restart  
 service quantum-l3-agent restart  
   
 )) 2>&1 | tee $0.log  
   

Ubuntu Kernel panic when creating Openstack instance

Currently I am using Openstack Folsom, nova version  2013.1.4, Installed on Ubuntu 12.04.  I upgraded my compute and network nodes's kernel to  "Linux 3.2.0-58-generic". Since then, when I try to create an instance on a the compute nodes or when I restart the network node, the systems goes to Kernel panic. 

The only fix for this issue I found, is to downgrade the kernel from "Linux 3.2.0-58-generic" to "Linux 3.2.0-55-generic" on all compute nodes and network node to keep things consistent.  The controller nodes does not seem to be affected by this kernel bug.

Also make sure to install the header if the corresponding kernel, if you are installing the kernel. Or if you have already got "Linux 3.2.0-55-generic" kernel installed, then just change it to the default kernel in grub records. 

First find out the index of the kernel in the grub records.

grep menuentry /boot/grub2/grub.cfg

Then the fist answer of the following question should help when editing the grub config.

http://askubuntu.com/questions/216398/set-older-kernel-as-default-grub-entry

Openstack Grizzily Error "FATAL: Module openvswitch_mod not found" when start openvswitch or at startup

I came across this error on Openstack Grizzily installed on Ubuntu 12.04 LTS, when I upgrade or degrade the linux kernel. So its very likely that you will get this error after running a "dist-upgrade" on your system.

The cause of this problem seem to be that when a different kernel is used, the openvswitch modules have not been built  with it.

Error:
FATAL: Module openvswitch_mod not found.
 * Inserting openvswitch module
 * not removing bridge module because bridges exist (virbr0)
invoke-rc.d: initscript openvswitch-switch, action "load-kmod" failed.

Solution:

Copy the following script to a file. Then save.

 #!/bin/bash  
   
 sudo apt-get install linux-headers-$(uname -r)  
   
 depmod  
 modprobe openvswitch  
   
 sudo apt-get remove openvswitch-switch openvswitch-datapath-dkms quantum-plugin-openvswitch-agent  
   
 sudo apt-get install openvswitch-switch openvswitch-datapath-dkms quantum-plugin-openvswitch-agent  
   
   

Then make it make the file executable.

Go to the file directory then execute following command in terminal:

sudo chmod +x filename

Finally execute the file.

./filename 

Openstack Folsom Error "FATAL: Module openvswitch_mod not found" when start openvswitch or at startup

I came across this error on Openstack Folsom installed on Ubuntu 12.04 LTS, when I upgrade or degrade the linux kernel. So its very likely that you will get this error after running a "dist-upgrade" on your system.

The cause of this problem seem to be that when a different kernel is used, the openvswitch modules have not been built  with it.

Error:

FATAL: Module openvswitch_mod not found.
 * Inserting openvswitch module
 * not removing bridge module because bridges exist (virbr0)
invoke-rc.d: initscript openvswitch-switch, action "load-kmod" failed.

Solution:

Copy the following script to a file. Then save.

 #!/bin/bash  
   
 sudo apt-get install linux-headers-$(uname -r)  
   
 sudo apt-get remove quantum-plugin-openvswitch openvswitch-switch quantum-plugin-openvswitch-agent openvswitch-datapath-dkms openvswitch-common quantum-common python-quantum  
   
 apt-get install openvswitch-switch  
 mkdir -p /etc/quantum/  
 apt-get install quantum-plugin-openvswitch-agent quantum-dhcp-agent quantum-l3-agent quantum-server  
   
 depmod  
 modprobe openvswitch  
   

Then make it make the file executable.

Go to the file directory then execute following command in terminal:

sudo chmod +x filename

Finally execute the file.

./filename 

No sound on Ubuntu 12.04 after upgrading from 11.10

I have recently upgraded my Ubuntu 11.10 distribution to Ubuntu 12.04 LTS on my HP pavillion dv6 laptop.


This is the output from 'aplay -l' command of my system.

card 0: Intel [HDA Intel], device 0: STAC92xx Analog [STAC92xx Analog]

  Subdevices: 0/1

  Subdevice #0: subdevice #0

card 0: Intel [HDA Intel], device 1: STAC92xx Digital [STAC92xx Digital]

  Subdevices: 1/1

  Subdevice #0: subdevice #0

card 2: HDMI [HDA ATI HDMI], device 3: HDMI 0 [HDMI 0]

  Subdevices: 1/1

  Subdevice #0: subdevice #0

After rebooting I noticed that I had no sound. After trying many things, to resolve this problem I had to add the following two lines to the "alsa-base.conf" file.

alias snd-card-0 snd-hda-intel

options snd-hda-intel model=auto

If this does not work then replace "auto" with the following keywords one at a time. Save the file and restart your computer each time you test with one configuration.

ref                    - Reference board

mic-ref            -Reference board with power management for ports

dell-s14           -Dell laptop

hp                     -HP laptops with (inverted) mute-LED

hp-dv7-4000  -HP dv-7 4000

auto                  - BIOS setup (default)

Reference :- ubuntu 12.04 no sound vostro 3300 i5
Chathura

Lunux/Ubuntu not booting up after the kernel update?

After I have upgraded to 2.6.38-9-generic kernel I got the following message and I have tried many thing but this is how I have solved the problem.

Gave up waiting for root device. Common problems:
-Boot args (cat /proc/cmdline)
-Check rootdelay= (did the system wait long enough?)
-Check root= (did the system wait for the right device?)
-Missing modules (cat /proc/modules; ls /dev)
ALERT! /dev/disk/by-uuid/310147a9-f21c-4e7c-aeb0-18f31073da58 does not exist. Dropping to a shell!

BusyBox v1.10.2 (Ubuntu 1:1.10.2-2ubuntu7) built-in shell (ash)
Enter 'help' for a list of built-in commands.

(initramfs)

Boot your computer and when the grub loader appears, select the linux version you use (to the rescue mode) and press 'e' to edit it.

Then remove the the UUID= part of the line from root=UUID=

Replace it with your current partition, so in my case it looks like

root=/dev/sda5

press +x to boot up with the changed settings

Then after you have logged in install fglrx module.

sudo apt-get install fglrx

Then restart.

Now this time select your linux version but this time not the rescue mode.

Then replace the UUID how you did earlier.

Done.

To permanently apply the settings to grub, you will have to edit the grub.cfg file

sudo vi /boot/grub/grub.cfg