How to Use contextmanager Decorator To Create Custom with Statements in Python
The aim of this page📝 is to describe the creation of a context manager on the example of a function that goes to a specified folder ⟹ executes a random function ⟹ returns to where it started — without having to handle the journey there and back each time manually
Context manager is a built-in Python protocol
implemented via the with
statement
- In Python, we have the
with
statement that replaces the need to use pair statements ...such asopen()
andclose()
for file handling - This is part of a protocol known as context manager
protocol is a set of operations that a type must support to implement
- For the behavior of context protocol, see ..With Statement < Compound statements < Python documentation ..PEP 343 — The “with” Statement | peps.python.org
- The essential behavior is that
context managers provide
__enter__()
and__exit__()
methods that are invoked on entry to and exit from the body of thewith
statement.
Context Library built-in module contains a contextmanager
decorator that can be used to implement a custom context manager
- context manager is also a function of context library module
@contextlib.contextmanager¶ This function is a decorator that can be used to define a factory function for
with
statement context managers, without needing to create a class or separate__enter__()
and__exit__()
methods.While many objects natively support use inwith
statements, sometimes a resource needs to be managed that isn’t a context manager in its own right, and doesn’t implement aclose()
method for use with contextlib.closing
- The structure of the custom context manager built with the
@contextmanager
decorator looks like this
- When in doubt about decorators see Explaining Function Decorators in Python
My implementation of the built-in context manager function
- In scripting/just CLI work (powershell/bash) it is typical that you need to go somewhere, do something, and return to the point of departure
- In PowerShell, this is done with the pair function
push-location
andpop-location
- This is the final implementation — ignore the first function that’s also a decorator I’m using to abstract away git for operational purposes …e.g. access provision to AWS subaccounts
- What matters here is the
push_and_pop_path
function and its structure which allows it to ...1.go to the repo, check if it's dirty, pull changes if needed, checkout the branch ...2.return ...3.run the function which in this case populatesiam
template for AWS access and copies the file to the repo with the checked-out branch ...4.to to the repo again, commit and push changes and ask if a PR should be opened
The whole story belongs to the wider epic of side-effects functions — functions, that come in pairs
- Uncle Bob has a nice rant about this
how good at we at managing pairs of functions like this?
for example how good are we at managing malloc and free
the answer to that is that we're terrible at managing that
the obvious evidence of our terrible ability to manage pairs of functions is that
in modern languages, we've invented a horrible hack to allow us to forget about managing pairs of functions
that horrible hack is called garbage collection
now I would not want to program without garbage collection
because garbage collection makes it much easier to write safe code
but you must admit that garbage collection is a crutch
it is not reasoned
you did not write the code in a reasoned way
you did not free everything you allocated
instead what you did is say - the system will take care of it
and ok fine we have written this horrible hack we've declared that we're
going to depend upon it we've acknowledged that we need the crutch
but allocation and freeing are not the only side effect function
there are many other side effect functions that you and I have to deal with
like open and close
does anybody seen a system crash because too many people forgot to close files
yes and you leave a bunch of file descriptors open in the operating system
and eventually, you run out of file descriptors that's called a file descriptor leak
has anybody seen a system crash because all the graphics contexts got leaked
or the semaphores didn't get closed
anything that comes in a pair like that will suffer the same fate that memory used to suffer
when we had memory leaks (and by the way you can still have memory leaks in Java)
— https://youtu.be/7EmboKQH8lM?t=5252