I really like how clever the algorithm you came up with is.
After facing a similar challenge, I discovered that the easiest way is to use the osmtogeojson tool (https://github.com/tyrasd/osmtogeojson) on the downloaded OSM data. It accepts xml and json inputs. Obviously, this is not pure Python, but if you can install it in your environment, it saves a lot of time and headache.
In order to get districts in Budapest without having to find the specific relations, I'd run the following Overpass query:
out skel qt;
and then pass this to osmtogeojson, and from then on, I can use my geo data in any GIS environment.