In the task manager, I can see that system spawned few python processes at once with some of the python processes that consume ~ 14GB of RAM. But sometimes I can see only 1 python process spawned that ...

Poetry Version: 1.2.1 Python: 3.10.5 Virtualenv Python: 3.10.5 Implementation: CPython Path: C:\Users\my_user\AppData\Local\pypoetry\Cache\virtualenvs\pyspark-write-demo-CwtTbJGc-py3.10 Executable: C: ...