IOzone for Arch Linux ARM

For different benchmarks I will later publish on this blog, I wanted to use IOzone. Unfortunately, this package is not available on Arch Linux ARM via the pacman package manager.

This tool is not difficult to compile, but still I decided to create an Arch Linux ARM package for it and share it with you: iozone-3_414-1-armv6h.pkg.tar.xz

You can install it using the following command:

pacman -U iozone-3_414-1-armv6h.pkg.tar.xz

Please note that I also tried to submit the package to the AUR (Arch User Repository). To my surprise, it was already present (https://aur.archlinux.org/packages/iozone/) but not available for ARM. 🙁

, , , , ,

No Comments

Android Side Navigation

The UI pattern “Side Navigation” (also called “Fly-In App Menu” or “Slide Out Navigation”) has been very famous lately. It started with Spotify and had shortly been followed by Evernote and Google+. It has now been adopted by a lot more applications such as YouTube, Facebook, Dribbble, etc.

Because of the great adoption of this navigation by the users, the company I am working for would like to implement it on one of their mobile applications.
A lot of libraries have been developed to meet this need! A good read would be Cyril Mottier’s posts about implementing this pattern:
The making of Prixing #1: Fly-in app menu
The making of Prixing #2: Swiping the fly-in app menu
The making of Prixing #3: Polishing the sliding app menu

All of this is good but which library to use?
To compare them, I built a sample application which includes all of these implementations:

If you wish to test it, you can download the APK file or go to the GitHub page.

The comparison has been a good way for us to determine which one would best fit our need. This is briefly the notes of our reflexion:

  • LibSlideMenu and RibbonMenu are rendering a Menu Resource instead of a Layout Resource which is too limited in our case;
  • SlidingMenu is only working with gesture (no OnClickListener implementation) on the contrary to android-fly-in-app-navigation which doesn’t work by gesture. However, we want to have the gesture and the OnClickListener implementation.

So, only two remains from our list of six libraries. Both of them are quite similar but we finally choose the android-menudrawer which has a slightly better design. 🙂

, , , , , , , , , , ,

No Comments

Retrieving UTF-8 values from Cassandra

I recently tried out Apache Cassandra which is a NoSQL solution that was initially developed by Facebook and designed to handle very large amounts of data spread out across many commodity servers while providing a highly available service with no single point of failure.

In order to populate the database, I used Apache Flume and Flume NG Apache Cassandra Sink which helped me to inject logs into it. But let’s focus on Cassandra here, I will write posts about Flume later on.

This is the Cassandra schema I was using (which is the one suggested by the Cassandra sink):

create keyspace logs with
   strategy_options = {datacenter1:1}
;

use logs;

create column family records with
   comparator = UTF8Type
   and gc_grace = 86400
;

After adding the data into the database, I wanted to fetch them to make sure everything went well.
I tried by three different ways:

  1. Cassandra CLI
    To return the first 100 rows (and all associated columns) from the records column family, I ran the following command:

    LIST records;

    However, the rows were looking like:

    RowKey: 6c6f67696b6465763a32653630306661362d633664652d346336612d386561352d323636326533353661616332
    => (column=data, value=39312e36362e3233392e323530202d202d205b32392f4465632f323031323a30393a31383a3433202d303730305d2022
    474554202f77702d636f6e74656e742f706c7567696e732f73796e746178686967686c6967687465722f73796e746178686967686c6967687465723
    22f736372697074732f636c6970626f6172642e73776620485454502f312e31222032303020313635392022687474703a2f2f7777772e6c6f67696b
    6465762e636f6d2f323031302f30372f30372f7573652d736572766572786d6c687474702d7468726f7567682d70726f78792f2220224d6f7a696c6
    c612f352e30202857696e646f7773204e5420362e313b20574f5736343b2072763a31372e3029204765636b6f2f3230313030313031204669726566
    6f782f31372e3022, timestamp=1357169131235135)
    => (column=host, value=39312e36362e3233392e323530, timestamp=1357169131235134)
    => (column=src, value=6c6f67696b646576, timestamp=1357169131235133)
    => (column=ts, value=323031322d31322d32395431363a31383a34332e3030305a, timestamp=1357169131235132) 
    

    This behavior is clearly explained on the DataStax page Getting Started Using the Cassandra CLI:

    Cassandra stores all data internally as hex byte arrays by default. If you do not specify a default row key validation class, column comparator and column validation class when you define the column family, Cassandra CLI will expect input data for row keys, column names, and column values to be in hex format (and data will be returned in hex format).
    To pass and return data in human-readable format, you can pass a value through an encoding function. Available encodings are:
    * ascii
    * bytes
    * integer (a generic variable-length integer type)
    * lexicalUUID
    * long
    * utf8

    Which means that we need to specify the encoding in which column family data should be returned. We can do it for the entire client session using the following commands:

    ASSUME records KEYS AS utf8;
    ASSUME records COMPARATOR AS utf8;
    ASSUME records VALIDATOR AS utf8;
    

    So if we now run the previous command, the rows are looking like:

    RowKey: logikdev:2e600fa6-c6de-4c6a-8ea5-2662e356aac2
    => (column=data, value=91.66.239.250 - - [29/Dec/2012:09:18:43 -0700] "GET /wp-content/plugins/syntaxhighlighter/syntax
    highlighter2/scripts/clipboard.swf HTTP/1.1" 200 1659 "http://www.logikdev.com/2010/07/07/use-serverxmlhttp-through-pro
    xy/" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20100101 Firefox/17.0", timestamp=1357169131235135)
    => (column=host, value=91.66.239.250, timestamp=1357169131235134)
    => (column=src, value=logikdev, timestamp=1357169131235133)
    => (column=ts, value=2012-12-29T16:18:43.000Z, timestamp=1357169131235132)
    

    Much better! 😉

  2. CQL
    In order to retrieve columns in the records column family, we can use the following SELECT command:

    SELECT * FROM records LIMIT 1;

    However, the output looks like:

     key                                                                                        | column1 | value
    --------------------------------------------------------------------------------------------+---------+---------------
    ----------------------------------------------------------------------------------------------------------------------
    ----------------------------------------------------------------------------------------------------------------------
    ----------------------------------------------------------------------------------------------------------------------
    ----------------------------------------------------------------------------------------------------------------------
    ----------------------------------------------------------------------------------------------------------------------
    ----------------------------------------------------------------------------------------------------------------------
    ----------------------------------------------------------------------------------------------------------------------
    -------------
     6c6f67696b6465763a31656438616166362d386438322d343330372d386133612d336438363664633133323636 |    data | 3131352e313131
    2e33322e313730202d202d205b32392f4465632f323031323a30353a34303a3338202d303730305d2022474554202f323031302f30382f32362f76
    616c69646174652d66696c652d73697a652d7072696f722d75706c6f61642f20485454502f312e31222032303020393931302022687474703a2f2f
    7777772e676f6f676c652e636f2e696e2f75726c3f73613d74267263743d6a26713d26657372633d7326736f757263653d7765622663643d392676
    65643d3043484951466a41492675726c3d687474702533412532462532467777772e6c6f67696b6465762e636f6d25324632303130253246303825
    3246323625324676616c69646174652d66696c652d73697a652d7072696f722d75706c6f61642532462665693d764f5465554c7157443457717241
    665f6d4948774177267573673d4146516a434e4839385070715373655375304e4a6a46484f534d5576515642704e41266361643d726a612220224d
    6f7a696c6c612f352e30202857696e646f7773204e5420362e313b2072763a31372e3029204765636b6f2f32303130303130312046697265666f78
    2f31372e3022
    

    As you can see, we have a similar problem than in Cassandra CLI. And funny enough, the resolution is also similar except that the syntax is a bit different.
    Indeed, to treat the column values in the table records as being of the type UTF-8, we need to run the following commands:

    ASSUME records(key) VALUES ARE text;
    ASSUME records(value) VALUES ARE text
    

    First of all, note that we overrode the type as ‘text’ instead of UTF-8 as explained on this page.
    Secondly, the cqlsh help page says we can simply run the command ASSUME records VALUES are text; which is supposed to treat all column values in the table records as being of the type text. But, for some reason, this is not working and we have to run it for each column value.

  3. twitter/cassandra
    Finally, I tried this Ruby client (which can be found here) directly with the Interactive Ruby Shell (IRB).
    The first thing we have to do is to connect to Cassandra using the following commands:

    require 'cassandra'
    client = Cassandra.new('logs', '127.0.0.1:9160')
    

    We then can run the command below to get one of the rows:

    client.get(:records, 'logikdev:69a2ef29-599f-48f0-99ef-d4c2ad1f4169')
    

    This would return something like the following:

    => #"85.168.229.99 - - [29/Dec/2012:09:48:22 -0700] \"GET /wp-content/plugins/syntaxhighlighter/s
    yntaxhighlighter2/styles/shThemeDefault.css?ver=2.1.364 HTTP/1.1\" 200 3936 \"http://www.logikdev.com/2010/12/02/update
    -your-twitter-status-with-java/\" \"Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20100101 Firefox/17.0\"", "host"
    =>"85.168.229.99", "src"=>"logikdev", "ts"=>"2012-12-29T16:48:22.000Z"}
    {"data"=>1357169131284005, "host"=>1357169131284004, "src"=>1357169131284003, "ts"=>1357169131284002}>
    

    And miracle!
    As you can see, there is no need to specify any type this time, the data are already decoded. 🙂

EDIT:
After some more research, I found a better solution to our problem: you can define the validation class in the create statement of the column family. It means that, for our example, the Cassandra schema would be:

create keyspace logs with
   strategy_options = {datacenter1:1}
;

use logs;

create column family records with
   comparator = UTF8Type
   and gc_grace = 86400
   and key_validation_class = UTF8Type
   and default_validation_class = UTF8Type
;

With this solution, there is no need to use the ASSUME command, the data will be displayed properly.

, , , , , , , , ,

3 Comments

MD5 generates 31 bytes instead of 32

I wrote the following method to generate MD5 hashes:

private static String md5(String input) throws NoSuchAlgorithmException {
    MessageDigest md = MessageDigest.getInstance("MD5");
    byte[] messageDigest = md.digest(input.getBytes());
    BigInteger number = new BigInteger(1, messageDigest);
    return number.toString(16);
}

This method takes a String and outputs a 32 character hash.

However, depending on the value of the parameter, this method outputs a 31 character hash instead of 32.
Why is that?

It appears that it is because the leading zero is missing. But the question remains! Why?
This is simply because the toString method doesn’t know that we are expecting a leading zero. BigInteger stores just a number; it doesn’t know how many leading zeroes we need.

For example, how many leading zero would you expect with the following code?

new BigInteger("33").toString()

In order to be sure MD5 generates 32 bytes, we could replace the number.toString(16) by:

String.format("%032x", number)

So, the method will look like the following:

private static String md5(String input) throws NoSuchAlgorithmException {
    MessageDigest md = MessageDigest.getInstance("MD5");
    byte[] messageDigest = md.digest(input.getBytes());
    BigInteger number = new BigInteger(1, messageDigest);
    return String.format("%032x", number);
}

, , , ,

No Comments

Remove DOS carriage return

After writing a shell script on Windows and trying to execute it on Linux, I got the following error message:

/bin/sh^M: bad interpreter: No such file or directory

The problem is obviously because the return line character on Windows and Linux are different. But how to fix it in order to execute it on Linux?
I firstly tried to open it, copy the content of the file and paste it in a new file using only Linux command line, but it didn’t work. 🙁

However, I found the following thread which fixed the problem:
http://www.reachdba.com/showthread.php?335-bin-sh-M-bad-interpreter-No-such-file-or-directory-apps11i-Instalation

Here is what you need to do:

  1. Open your file using vi
  2. Write the command :set fileformat=unix
  3. Save the file using :wq

The file should now run on Linux. 🙂

, , , , , , ,

1 Comment