Refactoring Django With FST
Django developers decided to drop Python 2 compatability in Django 2.0. There are serveral things that should be refactored/removed.
For example, in Python 2, programmers has to explicitly specify the class & instance when invoking
class Foo: def __init__(self): super(Foo, self).__init__()
In Python 3,
super can be invoked without arguments and it will choose right class & instance automatically.
class Foo: def __init__(self): super().__init__()
For this refactoring, a simple
sed search/replace should suffice. But, there are several hacks in codebase where super calls the grandparent instead of the parent. So,
sed won't work in such cases. Also it is very hard to refactor them manually and much harder for reviewers to review it as there are 1364 super calls in code base.
→ grep -rI "super(" | wc -l 1364
Changes has to be scripted. I wrote a simple script to replace super calls by class names. This worked only for 50% of the cases. It failed to capture classes which had comments on top of them, classes with decorators and nested classes.
To handle all these cases, a normal python script gets more complicated and there is no guarantee that it can handle all edge cases. So, I experimented with AST(Abstract Syntax Trees).
# this is a comment def foo(): print( "hello world" )
Converting above code to AST and then converting back gives this
def foo(): print('hello world')
Code to AST is a lossy transformation as they cannot preserve empty lines, comments and code formatting.
ast_to_code(code_to_ast(source_code)) != source_code
fst_to_code(code_to_fst(source_code)) == source_code
With RedBaron FST, just locate super calls, find nearest class node, check class name with super and replace accordingly. It took less than 10 lines of code.
RedBaron has good documentation with relveant examples and its API is similar to BeautifulSoup. Next time when writing code that modifies code consider using RedBaron.