2017

Automatic Magnetometer Calibration With Arduino


If we take readings from a 3-axis magnetometers like HMC5883L, AK8963C (used in MPU9250) or LSM303DLHC and plot them, its response should be a sphere with ceter at origin.

In practice, due to the presence of hard and soft iron distortions, the response will be an ellipsiod with its center shifted away from origin. We need to calibrate the magnetometer to nullify the distortions.

First we need to get sample readings of magnetometer in various positions. Depending on the magnetometer, we need to connect it to arduino and take readings by rotating it in 8 shape.

Calibration

Hard iron biases shifts center away from origin. We can eliminate this error by calculating the offsets and shifting the readings.

int mx, my, mz;

int mx_min, my_min, mz_min;
int mx_max, my_max, mz_max;
int mx_offset, my_offset, mz_offset;

int mx_calibrated, my_calibrated, mz_calibrated;

// get min/max values by taking readings
// from magnetometer of your choice

mx_offset = (mx_min + mx_max)/2;
my_offset = (my_min + my_max)/2;
mz_offset = (mz_min + mz_max)/2;

mx_calibrated = mx - mx_offset;
my_calibrated = my - my_offset;
mz_calibrated = mz - mz_offset;

Soft iron biases makes the axial responses uneven which results in ellipsiod shape. An easier way to correct this is to rescale the axial readings to an average value.

int mx_scale, my_scale, mz_scale;

mx_scale = (mx_max - mx_min)/2;
my_scale = (my_max - my_min)/2;
mz_scale = (mz_max - mz_min)/2;

float avg_scale = (mx_scale + my_scale + mz_scale)/3;

mx_calibrated = avg_scale/(mx - mx_offset);
my_calibrated = avg_scale/(my - my_offset);
mz_calibrated = avg_scale/(mz - mz_offset);

We can caclulate these biases once and store them in our code so that we don't need to calibrate it everytime. We can also write an auto update function which will recalibrate offsets & scale for every new reading.

Comments

Django Tips & Tricks #9 - Auto Register Models In Admin


Inbuilt admin interface is one the most powerful & popular feature of Django. Once we create the models, we need to register them with admin, so that it can read metadata and populate interface for it.

If the django project has too many models or if it has a legacy database, then adding all those models to admin becomes a tedious task. To automate this process, we can programatically fetch all the models in the project and register them with admin.

from django.apps import apps


models = apps.get_models()

for model in models:
    admin.site.register(model)

This works well if we are just auto registering all the models. However if we try some customisations and try to register them in admin.py files in our apps, there will be conflicts as Django doesn't allow registering the same model twice.

So, we need to make sure this piece of code runs after all admin.py files are loaded and it should ignore models which are already registered. We can safely hook this code in appconfig.

from django.apps import apps, AppConfig
from django.contrib import admin


class CustomApp(AppConfig):
    name = 'foo'

    def ready(self):
        models = apps.get_models()
        for model in models:
            try:
                admin.site.register(model)
            except admin.sites.AlreadyRegistered:
                pass

Now all models will get registed automatically. If we go to a model page in admin, it will just show 1 column like this.

This is not informative for the users who want to see the data. We can create a ListAdminMixin, which will populate list_display with all the fields in the model. We can create a new admin class which will subclass ListAdminMixin & ModelAdmin. We can use this admin class when we are registering the model so that all the fields in the model will show up in the admin.

from django.apps import apps, AppConfig
from django.contrib import admin


class ListAdminMixin(object):
    def __init__(self, model, admin_site):
        self.list_display = [field.name for field in model._meta.fields if field.name != "id"]
        super(ListAdminMixin, self).__init__(model, admin_site)


class CustomApp(AppConfig):
    name = 'foo'

    def ready(self):
        models = apps.get_models()
        for model in models:
            admin_class = type('AdminClass', (ListAdminMixin, admin.ModelAdmin), {})
            try:
                admin.site.register(model, admin_class)
            except admin.sites.AlreadyRegistered:
                pass

Now whenever we create a new model or add a new field to an existing model, it will get reflected in the admin automatically.

Comments

How To Install Custom ROMs In Xiamo MiPad?


Mi Pad and other Xiamo devices has Mi UI OS which is a dual boot system. A major problem with this system is it has only ~600 MB of space in 1st partition. Because of this, we cannot install some custom ROMs as they need more space. In this article we will see how to merge both partitions to get more free space and install any custom ROM.

Install TWRP

Dowload latest recovery from twrp.me. Copy it to android device or push it to android using adb.

adb push -p twrp-3.1.1-0-mocha.img /sdcard/

Now put the device into fastboot mode by pressing Volume down & Power button simultanesouly when you switch it on. When in fastboot mode, flash the downloaded recovery file

sudo fastboot flash recovery twrp-3.1.1-0-mocha.img

Now we can go to recovery mode using adb.

adb reboot recovery

Merge partitions

Before installing custom ROM, we need to merge partitions so that we will have enough space to install ROM. If you are familiar with parted command, you can directly merge the partitions from terminal in TWRP recovery.

There is also a script which you can flash to do the partition. You can read this guide on mi forum for more information.

After partition is completed, from TWRP go to Wipe -> Advanced wipe -> Select System -> Click on Repair or change file system. Here it should show that free space in system is more than 1GB.

Install ROM

After partitions are merged, it is straight forward to install any custom ROM. Download a custom ROM like lineage or RR, push it to device and then install it from TWRP. After reboot, you will see the custom ROM booting.

Comments

Django Tips & Tricks #8 - Hyperlink Foreignkey Fields In Admin


Consider Book model which has Author as foreignkey.

from django.db import models


class Author(models.Model):
    name = models.CharField(max_length=100)

class Book(models.Model):
    title = models.CharField(max_length=100)
    author = models.ForeignKey(Author)

We can register these models with admin interface as follows.

from django.contrib import admin

from .models import Author, Book

class BookAdmin(admin.ModelAdmin):
    list_display = ('name', 'author', )

admin.site.register(Author)
admin.site.register(Book, BookAdmin)

Once they are registed, admin page shows Book model like this.

While browing books, to go to a particular author, we have to previous page, go to Author model and then find relevant author. This becomes tedious if we spend lot of time in admin. Instead, if author field has a hyperlink, we can directly go to its page.

Django provides an option to access admin views by its URL reversing system. For example, we can get add view of author model in book app from reverse("admin:book_author_add").

To hyperlink author field in book admin, get url from reversing book_author_change with its id and return required html.

class BookAdmin(admin.ModelAdmin):
    list_display = ('name', 'author_link', )

    def author_link(self, book):
        link = reverse("admin:book_author_change", args=[book.author.id])
        return u'<a href="%s">%s</a>' % (link, book.author.name)
    author_link.allow_tags = True
    author_link.short_description = 'Author'

Now in the book admin view, author field will be hyperlinked and we can visit just by clicking it.

Update:

Django has inbuilt option for this. It provides list_display_links, to control which fields should be linked to change page. So, we can just add author field to it.

from django.contrib import admin

from .models import Author, Book

class BookAdmin(admin.ModelAdmin):
    list_display = ('name', 'author', )
    list_display_links = ('name', 'author',)

admin.site.register(Author)

Now, author field will be hyperlinked to its change page.

Comments

How To Remove Clock From LockScreen/StatusBar?


Last year, I wrote a blog post on how to remove clock from lock screen and status bar if xposed is installed on your android device. You can also do this without xposed if you are using RR(Resurrection Remix) as it comes with a lot of inbuilt customization.

Remove Clock From LockScreen

To remove clock from lockscreen, go to Settings -> Configurations -> Lock screen -> Show lock screen lock.

Remove Time From StatusBar

To remove clock from status bar, go to Settings -> Configurations -> Status bar -> System UI tuner -> Time -> Dont show this icon.

Once you do this, you will have a clean lockscreen and statusbar without any date or time them.

Comments

Bluetooth Serial Communication Between Ubuntu & Android


Most laptops and smart phones(Android/iPhone) have builtin Bluetooth modules. We can use this bluetooth module to communicate with each other or with other bluetooth modules like HC-05 or HM-10.

In this article, we will learn how to send data between laptop and android bluetooth.

First, we need to pair with a bluetooth device to send information. From Ubuntu, we can pair to a Bluetooth device from Bluetooth settings. Alternatively, we can also use CLI to do the same.

$ bluetoothctl
[NEW] Controller 24:0A:64:D7:99:AC asus [default]
[NEW] Device 94:E9:79:BB:F8:3A DESKTOP-C4ECO3K
[NEW] Device 88:79:7E:7B:4C:87 athene
[NEW] Device 94:65:2D:8C:2E:10 OnePlus 5
[NEW] Device 98:0C:A5:61:D5:64 Lenovo VIBE K5 Plus
[NEW] Device AC:C3:3A:A0:CE:EF Galaxy J2
[NEW] Device 98:D3:35:71:02:B3 HC-05

[bluetooth]# power on
Changing power on succeeded

[bluetooth]# agent on
Agent registered

[bluetooth]# default-agent
Default agent request successful

[bluetooth]# scan on
Discovery started
[CHG] Controller 24:0A:64:D7:99:AC Discovering: yes
[CHG] Device 94:E9:79:BB:F8:3A RSSI: -88
[CHG] Device 88:79:7E:7B:4C:87 RSSI: -66

[bluetooth]# pair 88:79:7E:7B:4C:87
Attempting to pair with 88:79:7E:7B:4C:87
[CHG] Device 88:79:7E:7B:4C:87 Paired: yes
Pairing successful

To communicate with paired devices, we will use RFCOMM protocol. RFCOMM is just a serial port emulation and provides reliable data tranfer like TCP.

From ubuntu, lets open a port for communication.

$ sudo rfcomm listen /dev/rfcomm0 3

From Android, we have to connect to ubuntu. For this, we can use Roboremo app which supports RFCOMM.

$ sudo rfcomm listen /dev/rfcomm0 3
Waiting for connection on channel 3
Connection from 88:79:7E:7B:4C:87 to /dev/rfcomm0
Press CTRL-C for hangup

Once the connection is established, we can communicate between devices.

In Unix like systems, OS provides a device file as an interface for device driver. To send and read messages from Linux or Mac is as easy as reading and writing to a file.

# to send message to bluetooth
$ echo 'hello from ubuntu' > /dev/rfcomm0

We can see the received messages on Android

We can also send messages from android and read from ubuntu.

# to read messages from bluetooth
$ cat /dev/rfcomm0
hello from android

This way, we can communicate with any bluetooth module using a laptop or a smart phone.

Comments

Amazon India Artificial Intelligence (AIAI) Summit


Today I have attended AIAI summit 2017 in Bangalore. Amazon started this summit to network with academia, industry data scientists and showcase amazon's work in machine learning.

There were 4 talks and 20 poster presentations. Most of them were scholars from IITB, IITM, IISC, IITH, one talk by CMU professor and remaining talks by Amazon.

Amazon team talked about the problems they are solving with machine learning like auto correcting address, suggesting right size for shoes/apparel, answering questions on products e.t.c.

People from academia talked about the research work they are doing and their results.

I have been working on Telugu OCR in the past few months. There were 3 poster presentations regarding indic languages. Even though a lot of papers were published in the past, there is no end-to-end OCR with good accuracy (>98%) for Indic languages. It is good to see Indic languages getting traction in deep learning.

Comments

Arduino Programming From Text Editor & CLI


To program Arduino, we can use Arduino IDE which facilitates writing and uploading code to the board.

If we are using a text editor for programming, it will have lot of customisations which speed up development process. In such case, it is better to use same text editor for arduino programming too.

I use Emacs as IDE and there is arduino mode for emacs which provides syntax highlighting and some useful utilites to write arduino code. We can find such packages for other editors also.

Arduino also provides cli interface to upload code to arduino. To upload code, we need to specify port, board and the code to upload.

In Linux system, to upload a file called foo.ino, we can run

arduino --port /dev/ttyACM0 --board arduino:avr:mega
 \ --upload foo.ino

An alternate way is to use platformio, an opensource tool chain for IoT development.

It can be installed using pip.

pip install platformio

Once it is installed, code can be directly uploaded using ci command.

pio ci --board=megaatmega2560 --project-option="upload_port=/dev/ttyACM0" --project-option="targets=upload" foo.ino

By this we can use text editor to write code and arduino/platformio to upload code to arduino board.

Comments

Using LSTM-CTC For Complex Script Recognistion


Most Indian languages have strong consonant-vowel structure which combine to give syllables. These syllables are written as one continuous ligature and they require complex text rendering (CTL) for type setting.

Writing OCR (Optical Character Recognistion) software for CTL scripts is a challenging task as segmentation is hard. Because of this overall accuracy drops drastically.

A better approach is to use Connectionist Temporal Classification (CTC) which can identify unsegmented sequence directly as it has one-to-one correspondence between input samples and output labels.

Here is a sample input and output of a RNN-CTC network which takes an unsegmented sequence and outputs labels.

Open source OCR software ocorpy uses BLSTM-CTC for text recognistion. Tesseract started using the same in its latest(4.0) version.

I have trained a model to recognize Telugu script using ocropy and the accuracy is ~99% which is far better when compared to OCR softwares without CTC which are accurate to ~70%.

Comments

Auto Register Subclasses Without Metaclass in Python


In registry pattern, a registry maintains global association from keys to objects, so that objects can be reached from anywhere by simple identifier. This is useful for doing reverse lookups.

When building a registry, programmers have to explicitly register each object with registry. Manually building a registry is error prone and it is tedious if there are too many objects to register. It is better to auto register objects if possible.

A commonly used approach is to use inheritance as an organizing mechanism. Create a meta class which will auto register classes and then create base class with this meta class.

REGISTRY = {}


def register_class(target_class):
    REGISTRY[target_class.__name__] = target_class


class MetaRegistry(type):

    def __new__(meta, name, bases, class_dict):
        cls = type.__new__(meta, name, bases, class_dict)
        if name not in registry:
            register_class(cls)
        return cls


class BaseClass(metaclass=MetaRegistry):
    pass


class Foo(BaseClass):
    pass


class Bar(BaseClass):
    pass

Now whenever you subclass BaseClass, it gets registered in the global registry. In the above example, Foo, Bar gets registered automatically.

Eventhough it solves registration problem, it is hard to understand the code unless you know how metaclasses work.

A simple alternative for this is to use __subclasses__() to get subclasess and register them.

REGISTRY = {cls.__name__: cls for cls in BaseClass.__subclasses__()}

This will work only for direct subclasses and won't with indirect subclasses like this.

class Baz(Bar):
    pass

To solve this, we can use a function to recursively retrieve all subclasses of a class.

def subclasses(cls, registry=None):
    if registry is None:
        registry = set()

    subs = cls.__subclasses__()

    for sub in subs:
        if sub in registry:
            return
        registry.add(sub)
        yield sub
        for sub in subclasses(sub, registry):
            yield sub


REGISTRY = {cls.__name__: cls for cls in subclasses(BaseClass)}

PEP 487 provides __init_subclass__ hook in class body to customize class creation without the use of metaclass. We can our registration logic in this __init_subclass__ hook.

class BaseClass:
    def __init_subclass__(cls, **kwargs):
        if cls not in registry:
            register_class(cls)
        super().__init_subclass__(**kwargs)

print(registry)

This is available only in Python 3.6+. For older versions, we have to use the recursive function to get all subclasess. This code is easier to understand than metaclass example.

Comments