November 20, 2014

Missing altgr keystrokes in VMware

If your VMWare keyboard mapping prevents you from using special keys such as Alt Gr, just add the following the your /etc/vmware/config:


















xkeymap.keycode.108 = 0x138 # Alt_R
xkeymap.keycode.106 = 0x135 # KP_Divide
xkeymap.keycode.104 = 0x11c # KP_Enter
xkeymap.keycode.111 = 0x148 # Up
xkeymap.keycode.116 = 0x150 # Down
xkeymap.keycode.113 = 0x14b # Left
xkeymap.keycode.114 = 0x14d # Right
xkeymap.keycode.105 = 0x11d # Control_R
xkeymap.keycode.118 = 0x152 # Insert
xkeymap.keycode.119 = 0x153 # Delete
xkeymap.keycode.110 = 0x147 # Home
xkeymap.keycode.115 = 0x14f # End
xkeymap.keycode.112 = 0x149 # Prior
xkeymap.keycode.117 = 0x151 # Next
xkeymap.keycode.78 = 0x46 # Scroll_Lock
xkeymap.keycode.127 = 0x100 # Pause
xkeymap.keycode.133 = 0x15b # Meta_L
xkeymap.keycode.134 = 0x15c # Meta_R
xkeymap.keycode.135 = 0x15d # Menu
 
 
Copied because the distribution customers @work needs this info, and I didn't relyon the resource being available. Thanks @

September 22, 2014

Bash one liners

Automatically remove dead "java" processes, simply replace process with java below

#ps aux | grep <process> | grep <optional other process> | tr -s " " | cut -d" " -f2 | xargs kill -9

The key point here is that the output from the ps command is cluttered with spaces and tabs when used together with cut. So, tr can be used to trim the lines, what the expression does is exactly that . It replaces all spaces with a single space character, allowing us to use this character as a delimiter to cut.

September 11, 2014

Testing memory maps on embedded systems

Here's the thing, sharing constant values between FPGA hardware (HW) and software (SW) can be hard for both sides. The constants that need sharing are memory addresses and masks for various registers. The main reason that sharing constants is hard is that no simple framework, at least to my knowledge, exist.

Usually, both sides of constants can be generated from some common format, or the HW side have to create a interface header containing the constants needed for the SW. Either way when a constant is changed it needs to tested properly.

But what happens when the HW side changes and the new FPGA code is released? Mayhem, since the HW doesn't have the complete SW source to test against, as their FPGA project could be in a different system, in a different source control etc.

In this case the HW should at least create a simple program that verifies their naming conventions, addresses and masks are the same as the last release. This is IMHO the smallest possible release test of a new FPGA image that can be run before releasing the HW. I'm not stating this is enough to actually test the HW, but it is enough to ensure that the SW does not break due to new names and wrong constants.

For this I have written a small C program. This program uses the MACROS defined by the HW and tests these. Firstly against C compilation, this step is important because changing a name would break compilation on the SW side, and changes like these should not be possible. Secondly, changing masks and address values should also be verified, at least against the previous version.

This will ensure that the HW side will know exactly what they break on the SW side if they change anything. And they'll clearly have a visual indication of what fails, enabling them to report any changes.

/***
  name: macro dumper
  version: 0.1
  author: john
  description:
  This program is a simple interface test for the defined memory map.
  The program dumps the macro's name and value to stdout.

  The program purpose is to test all generated macros against their generated
  name and value to ensure an interface test of the generated HW memory map

  The program will fail to compile any of the macros change their name according
  to the first version of the HW interface.

  The test program below is supposed to fail in the deliberate fail macro! This
  line may be commented in a release version of the simple test program

  Remember that new test cases must be added when new HW is introduced
  When you're introducing new macros in the hardware interface the following
  regular expressions will assist you in getting the macros from the interface
  header and into valuable C code that you can paste into the source's test
  sections below.

  The comment format change for a reason, namely the regular expression characters
  contain both * and / which is comment end ;)
***/
//  Regular expressions to:-
//  List all macros in the interface with all other things stripped:
//
//  egrep "#define [a-zA-Z0-9_]+ +" macro_dumper.c | perl -p -e "s/#define +(.+) \((.*)\)/NAME: \$1 VALUE: \$2/g"
//
//  Replace all macros with a corresponding test call for inserting into the name test:
//
//  egrep "#define [a-zA-Z0-9_]+ +" macro_dumper.c | perl -p -e "s/#define +(.+) \((.*)\) ?.*/TEST_NAME(\$1);/"
//
//  Replace all macros with a corresponding test call for inserting into the value test:
//
//  egrep "#define [a-zA-Z0-9_]+ +" macro_dumper.c | perl -p -e "s/#define +(.+) \((.*)\) ?.*/TEST_VALUE(\$1,\$2);/"
#include <stdio.h>

/*
  This test result construction is meant for the test to produce an output for any other command line tool.
  The purpose is that the test_result returns >0 in case of errors and report these to the OS.

  A return value of zero indicates a failed test
*/
static int override_return_value = 0;
static int test_result = 0;
static int test_case = 0;
int assign_test_result(int value,int varname)
{
    if(value != varname) {
    test_result++;
    return 0;
    }
    return 1;
}

#define TEST_NAME(varname) fprintf(stdout, "%s = %x\n", #varname, (varname));
#define TEST_VALUE(varname,value) fprintf(stdout, "%d - %s = %s\n",++test_case, #varname, (assign_test_result(varname,value))?"ok":"error");

/* #include "memorymap.h" */
/* The macros below are ment to be placed in the memorymap file provided by HW, here they are simply a test of the concept */
#define TEST1_MACRO (0x010000000)                 /* emulating some start address */
#define TEST2_MACRO (0x2000)                      /* emulating some offset */
#define TEST3_MACRO (TEST1_MACRO + TEST2_MACRO)   /* Check to see that the macro names are reported correctly when combined */
#define TEST_MACRO_MASK (0x0010)                  /* Test for some kind of mask */
#define DELIBERATE_ERROR_MACRO (0xff)             /* The check for this macro should not be 0xff as we deliberately want this check to fail */

int main(int argc, char** argv)
{
    override_return_value = argc-1;

    fprintf(stdout,"%s","This program checks the name and address values of memory map macros\n");
    if(override_return_value) {
    fprintf(stdout,"%s","Running in override return value mode, errors are only reported not breaking!\n");
    }

    /*
      This is the name test part of the program, any non complying marcos will cause a compilation error
      on the exact line of the non existing macro name. The test is valuable for finding non compatible
      name changes in the interface
    */
    TEST_NAME(TEST1_MACRO);
    TEST_NAME(TEST2_MACRO);
    TEST_NAME(TEST3_MACRO);
    TEST_NAME(TEST_MACRO_MASK);
    TEST_NAME(DELIBERATE_ERROR_MACRO);

    /*
      This part tests against the original macro values to ensure that we know when memory addresses actually change
      This test is to ensure that we know when a macro value changes. This is not a test to break the interface
      It's simply an indication that an address or mask has changed.
    */
    TEST_VALUE(TEST1_MACRO,0x010000000);
    TEST_VALUE(TEST2_MACRO,0x2000);
    TEST_VALUE(TEST3_MACRO,0x010000000 + 0x2000);
    TEST_VALUE(TEST_MACRO_MASK,0x0010);
    TEST_VALUE(DELIBERATE_ERROR_MACRO,4); /* This is a deliberate error to test the test macro ;) */

    /*
       Reporting test case statistics
     */
    fprintf(stdout,"Tests run: %d\n",test_case);
    fprintf(stdout,"Failures: %d\n",test_result);
    return (override_return_value)?test_result:0;
}



The program is compiled using gcc or equivalent by issuing:

john@BlackWidow gcc macro_dumper.c -o macro_dumper

In your terminal. You run the program by issuing:

john@BlackWidow ~/john/src/c $ ./macro_dumper
This program checks the name and address values of memory map macros
TEST1_MACRO = 10000000
TEST2_MACRO = 2000
TEST3_MACRO = 10002000
TEST_MACRO_MASK = 10
DELIBERATE_ERROR_MACRO = ff
1 - TEST1_MACRO = ok
2 - TEST2_MACRO = ok
3 - TEST3_MACRO = ok
4 - TEST_MACRO_MASK = ok
5 - DELIBERATE_ERROR_MACRO = error
Tests run: 5
Failures: 1
john@BlackWidow ~/john/src/c $ echo $?
0


Above you can see the output from the program. Keep in mind that the program should be compiled at every HW release and run to check against the interface.

The program is constructed in such a way that giving 1 or more arguments of any type will cause the program to report test case failure to the OS meaning that it could be used to break a Jenkins build bot on failures.

Yes it can be expanded. Yes it can be re factored. Yes there's lots of potential for improvement. But for now it's the smallest possible thing that will safe many hours of non working interfaces if something went wrong.

September 09, 2014

Matplotlib on mint 13

Today, I had a friend who needed to use matplotlib on our company linux boxes. We have some issues with out proxy server that rules python pip out. This mixed with python version and other dependencies lead me to write this small script, as a kind of configure script, for his code project.

The script deals with installing python3 and other prebuilt packages for ubuntu (mint) and then uses python3 build mechanisms & easy_install to add any remaining python egg dependencies.

The script source & listing:
#!/bin/sh
#
# This is the friendly python3 matplotlib installer script
# it checks for all matplotlib dependencies as pr. Sep 9 2014
# Any additions to the installer must be added by hand!
#
# If none of the dependencies are installed, the script will:
# 1. install prebuild ubuntu packages
# 2. Create a local dir for downloaded files
# 3. install dependencies by downloading these from various sources
# 3.a cython
# 3.b numpy, scipy
# 3.c
pyttk, six, tkinter, pip,matplotlib
# 4. remove the directory after installation
#
# john


# 1.
sudo aptitude install python3 python3-dev python3-setuptools python3-tk python3-cairo python3-cairo-dev libpng-dev
sudo aptitude install gfortran libopenblas-dev liblapack-dev
sudo easy_install3 -U distribute

# 2.
matlib=/tmp/matlib
[ -d $matlib ] && sudo rm -rf $matlib
mkdir $matlib && cd $matlib

# 3. Install dependencies, these vary and will change for furure releases!
# As the matplotlib we use in the dse require bleeding edge, we'll grap the latest stuff from the bleeding repositories

# 3.a
wget http://cython.org/release/Cython-0.20.2.tar.gz && tar -zxvf Cython-0.20.2.tar.gz && cd Cython-0.20.2/ && sudo python3 setup.py install && cd -

# 3.b
#install bleeding edge eggs from git repositories
for package in numpy scipy; do
    [ -d $package ] && sudo rm -rf $package
    git clone http://github.com/$package/$package.git $package && cd $package && sudo python3 setup.py install && cd -
done

# 3.c
# install bleeding edge eggs using easy_install
# NOTICE: this loop can only be extented if the egg name is the same in both the python code and module name in bash!
for egg in pip six tkinter pyttk matplotlib; do
    sudo easy_install3 $egg
done


pip freeze

cd -

# 4.
sudo rm -rf $matlib

March 31, 2014

Configuring VNC connections to you Box

A quick entry today. I needed to rid the *missing ubuntu session* dialog when connecting via VNC to my box@work.

So, I edited the vnc xstartup file:

#!/bin/sh
unset SESSION_MANAGER
gnome-session --session=gnome-classic &

[ -x /etc/vnc/xstartup ] && exec /etc/vnc/xstartup
[ -r $HOME/.Xresources ] && xrdb $HOME/.Xresources
xsetroot -solid grey
vncconfig -iconic &


Finally start the vncserver with something like:

#vncserver -geometry 1280x1024 -depth 24

In depth resource for this issue.

March 04, 2014

Assogiate

But Mooooooom it wan't it like window$. Me just pointy clicky and then like magic the file i clicked opens in my favorite editor! Sigh... Did you ever have programmers (users) that worked their entire career on a single platform. Programmers that think everything should work like it used to... Programmers unwilling to learn new stuff?

I have a bunch of programmer rednecks who apparently need to point and click in "the configured file manager" to be able to open a source file. And these programmers want the same behavior on Linux as they have in Window$ file Exploder ... Sigh ...

Here's how: http://ubuntugenius.wordpress.com/2009/11/19/create-your-own-file-types-in-ubuntu-with-assogiate/.  In my case I had to create a debian package carrying the settings, enabling these programmers to use synaptic to install the extension.

Programmers? ... Go figure ..



February 25, 2014

How to Remove the Visual Deals Popup Spam

I normally don't do this, but these deals are so annoying, so here's: How to Remove the Visual Deals Popup Spam, its hidden in your pinterest button!

February 10, 2014

gdb tips

I usually debug lLnux programs using gdb, I like the neatness of the tui mode, thanks for that feature! Here are some simple commands to get you started using gdb. There are several major doc out there, this is simply meant as a quick start and remembering reference for me, I'll make no attempt to replace any of these.
  • Starting your program with arguments: gdb -tui --args love_calc john jane
  • Setting a breakpoint in a specific file (in your huge project): b filename.cpp:linenum, i.e. b posix_thread.cpp:25
  • Stepping is s or (step) stepping into is stepi and continue is c.
  • Examining variables is done with p variable_name, i.e. p argv[1]
The above image shows the result of the commands.

There's a small guide from gnu on tui single key mode & a quick guide if you need more than I have shown here.





Typesafe bit mask operations or Bits, bytes and operators.

In production? You're kidding right? No way some network structures contain c bit-fields. Turns out he wasn't kidding (sigh). Sometimes you just have to wonder how, or who, that stuff get's in there.

There's a neat trick in c++ for creating type safe bitmasks using enums and a template class. You'll have to know your C++ operators and how to override these if you're looking for more than just my simple template class. The source is here. You build it with: g++ bitfiled.cpp -o bitfield -Werror

#include <stdio.h>
#include <stddef.h>
#include <stdint.h>
#include <iostream>
#include <bitset>

template<class T,typename S>
class bitfield {
public:

    inline bitfield();
    inline bitfield(const T &bit);

    inline const size_t size() const;
    inline const S num_bits() const;
    inline const S get(const T &bit) const;
    inline const S &get_bits() const;

    inline void set(const T &bit);
    inline void clear(const T &bit);
    inline void toggle(const T &bit);

    inline S operator^=(const T &bit);
    inline S operator|=(const T &bit);
    inline S operator&(const T &bit);
    inline S operator=(const T &bit);

    inline const char *dump()const;

private:    
    S bits;
    static const S zero = 0;
    static const S bit_base = 8;
};

template<typename T,typename U>
inline void set_bit(U &bits,const T mask)
{
    bits |= mask;
}

template<typename T,typename U>
inline void toggle_bit(U &bits,const T mask) 
{
    bits ^= mask;
}

template<typename T,typename U>
inline uint8_t clear_bit(U &bits,const T mask) 
{
    return bits &= ~mask;
}

template<typename T,typename U>
inline const bool is_bit_set(const U &bits,const T mask)
{
    return bits & mask;
}

template<class T,typename S>
inline bitfield<T,S>::bitfield()
{
    bits = zero;
}

template<class T,typename S>
inline bitfield<T,S>::bitfield(const T &bit)
{
    bits = bit;
}

template<class T,typename S>
inline const S &bitfield<T,S>::get_bits() const
{
    return bits;
}

template<class T,typename S>
inline const size_t bitfield<T,S>::size() const
{
    return sizeof(*this);
}

template<class T,typename S>
inline const S bitfield<T,S>::num_bits() const
{
    return size()*bit_base;
}

template<class T,typename S>
inline void bitfield<T,S>::set(const T &bit) 
{
    ::set_bit(bits,bit);
}

template<class T,typename S>
inline void bitfield<T,S>::clear(const T &bit)
{
    ::clear_bit(bits,bit);
}

template<class T,typename S>
inline const S bitfield<T,S>::get(const T &bit) const
{
    return ::is_bit_set(bits,bit);
}

template<class T,typename S>
inline void bitfield<T,S>::toggle(const T &bit)
{
    ::toggle_bit(bits,bit);
}

template<class T,typename S>
inline const char *bitfield<T,S>::dump() const
{
    std::string out;
    for(unsigned int i=num_bits();0!=i;i--)
    {
out += ((1 << (i-1)) & bits) ? "1" : "0";
    }
    return out.c_str();
}

template<class T,typename S>
inline S bitfield<T,S>::operator^=(const T &bit)
{
    ::toggle_bit(bits,bit);
    return bits;
}

template<class T,typename S>
inline S bitfield<T,S>::operator|=(const T &bit)
{
    ::set_bit(bits,bit);
    return bits;
}

template<class T,typename S>
inline S bitfield<T,S>::operator&(const T &bit)
{
    return ::is_bit_set(bits,bit);
}

template<class T,typename S>
inline S bitfield<T,S>::operator=(const T &bit)
{
    return bits = bit;
}

enum Mask16 {
    ab1=0x0001,
    ab2=0x0002,
    ab3=0x0004,
    ab4=0x0008,
    ab5=0x0010,
    ab6=0x0020,
    ab7=0x0040,
    ab8=0x0080,
    ab9=0x0100,
    ab10=0x0200,
    ab11=0x0400,
    ab12=0x0800,
    ab13=0x1000,
    ab14=0x2000,
    ab15=0x4000,
    ab16=0x8000,
};

enum Mask8 {
    b1 = 0x01,
    b2 = 0x02,
    b3 = 0x04,
    b4 = 0x08,
    b5 = 0x10,
    b6 = 0x20,
    b7 = 0x40,
    b8 = 0x80,
};

int main (int argc, char **argv)
{
    bitfield<Mask8,uint8_t> bf8;
    std::cout << "-------------------------------" << std::endl;
    std::cout << "bf8 size: " << bf8.size() << std::endl;
    std::cout << "Bits constructor: " << std::bitset<8>(bf8.get_bits()) << std::endl;

//    bf8.set(b2);
    bf8 |= b2;
    std::cout << "Bit initialized: " << std::bitset<8>(bf8.get_bits()) << std::endl;

    uint8_t bit_flip = 0;
    for(int i=0;i<8;i++)
    {
bit_flip = (1 << i); 
const char *p = (bit_flip<=0x08) ? "0x0" : "0x";
std::cout << "-------------------------------" << std::endl;
std::cout << "Simulated Mask: " << std::bitset<8>(bit_flip) << std::endl;
std::cout << "Simulated Hex : " << p << std::hex << (int)bit_flip << std::endl;

// bf8.toggle(static_cast<Mask8>(bit_flip));
bf8 ^= static_cast<Mask8>(bit_flip);
// (bf8.get(static_cast<Mask8>(bit_flip))) ? std::cout << "true" << std::endl :std::cout << "false" << std::endl ;
(bf8 & static_cast<Mask8>(bit_flip)) ? std::cout << "true" << std::endl :std::cout << "false" << std::endl ;
std::cout << "bf8.bits " << std::bitset<8>(bf8.get_bits()) << std::endl;
    }

    bitfield<Mask16,uint16_t> bf16;
    std::cout << "-------------------------------" << std::endl;
    std::cout << "bf16 size: " << bf16.size() << std::endl;
    std::cout << "Bits constructor: " << std::bitset<16>(bf16.get_bits()) << std::endl;

    bf16.set(ab9);
    std::cout << "Bit initialized: " << std::bitset<16>(bf16.get_bits()) << std::endl;

    bf16 = ab10;
    std::cout << "Bit initialized: " << std::bitset<16>(bf16.get_bits()) << std::endl;
    std::cout << "num bits: " << std::dec << bf16.num_bits() << std::endl;

    // testing for placement in a telegram!
    struct test_telegram {
uint8_t version;
uint8_t type;
bitfield<Mask8,uint8_t> b;
    }tt = {0};

    std::cout << "-------------------------------" << std::endl;
    std::cout << "tt size: " << sizeof(tt) << std::endl;

    tt.b = b3;
    std::cout << "tt.b: " << std::bitset<8>(tt.b.get_bits()) << std::endl;
    std::cout << "tt.b.dump() : " << tt.b.dump() << std::endl;

    bitfield<Mask8,uint8_t> bf_constructor(b5);
    std::cout << "-------------------------------" << std::endl;
    std::cout << "bf_constructor: " << std::bitset<8>(bf_constructor.get_bits()) << std::endl;
    std::cout << "bf_constructor.dump() : " << bf_constructor.dump() << std::endl;

    // Using the template function to manipulate c style!
    ::set_bit(bf_constructor,b3);
    std::cout << "global function - bf_constructor: " << std::bitset<8>(bf_constructor.get_bits()) << std::endl;

//    ::set_bit(bf_constructor,0x08); // error needs a valid Mask8 type or a cast static_cast<Mask8>(0x08)
//    bf_constructor.set(0x08);       // error needs a valid Mask8 type or a cast static_cast<Mask8>(0x08)
//    bf_constructor.get(0x08);       // error needs a valid Mask8 type or a cast static_cast<Mask8>(0x08)
//    bf_constructor.toggle(0x08);    // error needs a valid Mask8 type or a cast static_cast<Mask8>(0x08)
    return 0;
}

I created 4 template functions, set_bit, clear_bit, toggle_bit and is_bit_set. Their purpose is to both serve as a simple interface, and to keep the types in place. They're used through the class, and they could be declared in the class, but for me that would make the functions loose their purpose. 

Notice there are two enumerations, namely, Mask8 and Mask16. These two types are used in the bitfield to ensure the type safety. The main function is meant as a simple test of the type safety. Play with this stuff all you wan't.

You'll notice that you cannot set, say, an int in either of the mask operators (or functions) all will give you a compile error, the sole intent of the implementation. Where do you use it you ask, I'd be using it every where I find a c/c++ bitfield used inside a structure, to keep the code portable on many platforms and to be able to use the above bitfield class in network telegrams.