Note: This was originally posted at martinheinz.dev
There are plenty of articles written about lots of cool features in Python such as variable unpacking, partial functions, enumerating iterables, but there is much more to talk about when it comes to Python, so here I will try to show some of the features I know and use, that I haven’t yet seen mentioned elsewhere. So here we go.
Sanitizing String Input
Problem of sanitizing user input applies to almost every program you might write. Often it’s enough to convert characters to lower or upper-case, sometimes you can use Regex to do the work, but for complex cases, there might be a better way:
In this example you can see that whitespace characters “\n” and “\t” have been replaced by single space and “\r” has been removed completely. This is a simple example, but we could take it much further and generate big remapping tables using unicodedata package and its combining() function to generate and map which we could use to remove all accents from string.
Taking Slice of an Iterator
If you try to take slice of an Iterator, you will get a TypeError, stating that generator object is not subscriptable, but there is a easy solution to that:
Using itertools.islice we can create a islice object which is an iterator that produces desired items. It's important to note though, that this consumes all generator items up until the start of slice and also all the items in our islice object.
Skipping Begining of Iterable
Sometimes you have to work with files which you know that start with variable number of unwanted lines such as comments. itertools again provides easy solution to that:
This code snippet produces only lines after initial comment section. This approach can be useful in case we only want to discard items (lines in this instance) at the beginning of the iterable and don’t know how many of them there are.
Functions with only Keyword Arguments (kwargs)
It can be helpful to create function that only takes keyword arguments to provide (force) more clarity when using such function:
As you can see this can be easily solved by placing single * argument before keyword arguments. There can obviously be positional arguments if we place them before the * argument.
Creating Object That Supports with Statements
We all know how to, for example open file or maybe acquire locks using with statement, but can we actually implement our own? Yes, we can implement context-manager protocol using __enter__ and __exit__ methods:
This is the most common way to implement context management in Python, but there is easier way to do it:
The snippet above implements the content management protocol using contextmanager manager decorator. The first part of the tag function (before yield) is executed when entering the with block, then the block is executed and finally rest of the tag function is executed.
Saving Memory with __slots__
If you ever wrote a program that was creating really big number of instances of some class, you might have noticed that your program suddenly needed a lot of memory. That is because Python uses dictionaries to represent attributes of instances of classes, which makes it fast but not very memory efficient, which is usually not a problem. However, if it becomes a problem for your program you might try using __slots__:
What happens here is that when we define __slots__ attribute, Python uses small fixed-size array for the attributes instead of dictionary, which greatly reduces memory needed for each instance. There are also some downsides to using __slots__ - we can't declare any new attributes and we are restricted to using ones on __slots__. Also classes with __slots__ can't use multiple inheritance.
Limiting CPU and Memory Usage
If instead of optimizing your program memory or CPU usage, you want to just straight up limit it to some hard number, then Python has a library for that too:
Here we can see both options to set maximum CPU runtime as well as maximum memory used limit. For CPU limit we first get soft and hard limit for that specific resource ( RLIMIT_CPU) and then set it using number of seconds specified by argument and previously retrieved hard limit. Finally, we register signal that causes system exit if CPU time is exceeded. As for the memory, we again retrieve soft and hard limit and set it using setrlimit with size argument and retrieved hard limit.
Controlling What Can Be Imported and What Not
Some languages have very obvious mechanism for exporting members (variables, methods, interfaces) such as Golang, where only members starting with upper-case letter are exported. In Python on the other hand, everything is exported, unless we use __all__:
Based on code snippet above, we know that only bar function will be exported. Also, we can leave __all__ empty and nothing will be exported casing AttributeError when importing from this module.
Comparison Operators the Easy Way
It can be pretty annoying to implement all the comparison operators for one class, considering there are quite a few of them — __lt__ , __le__ , __gt__ , or __ge__. But what if there was an easier way to do it? functools.total_ordering to the rescue:
How does this actually work? total_ordering decorator is used to simplify the process of implementing ordering of instances for our class. It's only needed to define __lt__ and __eq__, which is the minimum needed for mapping of remaining operations and that's the job of decorator - it fills the gaps for us.
Not all these features are essential and useful in day-to-day Python programming, but some of them might come in handy from time to time and they also might simplify task that would be otherwise quite lengthy and annoying to implement. Also I want to note that all those features are part of Python standard library, while some of them seem to me like pretty non-standard things to have in standard library, so whenever you decide to implement something in Python first go looking for it in standard library and if you can’t find it, then you are probably not looking hard enough (if it’s really not there, then it’s surely in some third party library). 🙂