Benchmarking Android/Linux Device Driver

Recently, I did benchmarking for an Android and Linux device driver.

While benchmarking it is important to give consideration to:

1. What you need to benchmark?
2. What is the correct approach you are taking for it?
3. What APIs/Tools you are using for measuring various metrics?

1. What you need to benchmark?:
   The Purpose and Metrics

In my case I was required to benchmark the performance of a default Android SDIO UART driver against the custom made SDIO UART driver. To be specific the requirement was to measure the time taken to read & write the data.

2. What is the correct approach you are taking for it? :
   The Right Approach

There were different possibilities to go for it, like:

• Use time command and find out total time taken between invocation and termination of read/write operation and compare its result for both default and custom driver. I found the output of time was not that accurate as it considers other variables (user CPU time, system CPU time, etc.) which I felt was not a good idea.
• Use a kernel level API just before the driver’s read/write implementation to find out exact time spent in those operations. This was closer to what I wanted in my benchmarking effort.
3. What APIs/Tools you are using for measuring various metrics? :
   Well analyzed, understood APIs/Tools

Next decision was on which Kernel API to use for benchmarking:

• Use current_kernel_time() just before and after the read/write implementation to find precisely how much time was spent in execution.
  

  static void sdio_uart_receive_chars(struct sdio_uart_port *port, unsigned int *status)
  {
  #ifdef SDIO_UART_DEBUG
  struct timespec time_spec1, time_spec2;
  time_spec1 = current_kernel_time();
  #endif

  /* code for read the data */

  #ifdef SDIO_UART_DEBUG
  time_spec2 = current_kernel_time();
  printk(KERN_INFO "\n MY_DBG : read took: %ld nanoseconds",
  (time_spec2.tv_sec - time_spec1.tv_sec) * 1000000000 + (time_spec2.tv_nsec - time_spec1.tv_nsec));
  #endif

  }

  Then found, most of the times it was showing 0 nanoseconds for read/write and very few times it showed some values in nanoseconds, which was not sure if those time values were accurate.

• Then finally as per a perfect suggestion from a wise friend, shodanex, I used getnstimeofday().
  

  static void sdio_uart_receive_chars(struct sdio_uart_port *port, unsigned int *status)
  {
  #ifdef SDIO_UART_DEBUG
  struct timespec time_spec1, time_spec2;
  getnstimeofday(&time_spec1);
  #endif

  /* code for read the data */

  #ifdef SDIO_UART_DEBUG
  getnstimeofday(&time_spec2);
  printk(KERN_INFO "\n MY_DBG : read took: %ld nanoseconds",
  (time_spec2.tv_sec - time_spec1.tv_sec) * 1000000000 + (time_spec2.tv_nsec - time_spec1.tv_nsec));
  #endif

  }

Then I could get what I wanted, a true comparison in nanoseconds for driver’s read/write operation.

Following is the benchmarking result (partial) for read() operation.

I think similar approach can be followed to benchmark the device drivers on other target platforms.

Finalizing correct metrics is very essential as at times it can distort the purpose of the benchmarking process. Choosing well-thought approach for benchmarking always helps to make the result more meaningful and the resources well-spent! Finally, as they say more time you spent on sharpening the tools, the less time you spend in cutting the wood! In benchmarking, more time you spend in analyzing the right API/tool for your requirement, less you waste the time & energy while benchmarking and more you add value to it!

All the best!