Python package importing explained

Python is a powerful and elegant language which makes people obsessed into it (such as me). However, many of the python users are still confused by two question:

1. package importing
2. encoding and decoding

This article will explained the package importing with some simple examples.

There are two type of importing in python:

1. Relative importing
2. Absolute importing

Relative importing

Just like *nix system, . is used for relative path. One example is:

└── source
    ├── __init__.py
    ├── __init__.pyc
    ├── bar
    │   ├── __init__.py
    │   ├── __init__.pyc
    │   ├── bar.py
    │   └── bar.pyc
    ├── foo
    │   ├── __init__.py
    │   ├── __init__.pyc
    │   ├── foo.py
    │   └── foo.pyc
    └── main.py

Example 1:

# foo.py
print("foo.py's __name__: {}".format(__name__))
print("foo.py's __package__: {}".format(__package__))
from ..bar import bar
b = "foo " + bar.a


# bar.py
print("bar.py's __name__: {}".format(__name__))
print("bar.py's __package__: {}".format(__package__))
a = "bar"



# main.py
print("main.py's __name__: {}".format(__name__))
print("main.py's __package__: {}".format(__package__))
from foo import foo

if __name__ == "__main__":
    print(foo.b)

.. is to tell foo.py to find the variable from above and bar directory. However, when run:

cd source
python main.py

Error is given out:

Traceback (most recent call last):
  File "/Volumes/Apple/ben_ws/python_dev/packageImporting/source/main.py", line 3, in <module>
    from foo import foo
  File "/Volumes/Apple/ben_ws/python_dev/packageImporting/source/foo/foo.py", line 3, in <module>
    from ..bar import bar
ValueError: Attempted relative import beyond toplevel package

How would this happen? Before solving this issue, it is a must to know each of the python scripts in the project does have a __name__ and a __package__ attribute which are used to identify their position in whole package/project. In any script, if the __name__ equals __main__, it tells the interpreter current script is the top level script, we can understand this concept as the base position.  All the other scripts are relative to the main script by positioning __package__.

In the above example, if we print out __name__ and __package__ for each of the script:

As you can see, three scripts have no way to know each other's existence. Fortunately PEP 0366 -- Main module explicit relative imports introduced __package__ to make isolated scripts joint together:

cd .. # jump out the source dir
python -m source.main

By assigning a top level root package to main.py (by using -m) and so all other scripts can be relative to, we now have:

Now both foo.py and bar.py know how to refers to main.py ---- by __package__!

Absolute importing

Absolute importing is not supported by python until python2.5. So in the earlier version of other python project code, you might see from __future__ import absolute_import as work-around to make this feature available. Now Absolute importing is fully supported by python2.6 and python3 (and above) and they are even the default importing method in those versions.

Same structure with last example but with Absolute importing:

Example 2:

# foo.py
print("foo.py's __name__: {}".format(__name__))
print("foo.py's __package__: {}".format(__package__))
from source.bar import bar  # Different here!
b = "foo " + bar.a


# bar.py
print("bar.py's __name__: {}".format(__name__))
print("bar.py's __package__: {}".format(__package__))
a = "bar"



# main.py
print("main.py's __name__: {}".format(__name__))
print("main.py's __package__: {}".format(__package__))
from source.foo import foo  # Different here!

if __name__ == "__main__":
    print(foo.b)

cd source
python main.py

Error happened again!:

Traceback (most recent call last):
  File "main.py", line 3, in <module>
    from source.foo import foo
ImportError: No module named source.foo

if check the __name__ and __package__ again:

We cannot get the other two scripts' __name__ and __package__ as the import error happened. Very similar to the solution on example 1, assign a package to main.py:

python -m source.main

Conclusion

Use Absolute importing is recommended for any of your projects as it brings you clean structure, easier for your to update the impotring if you have multiple local importing.

If you have any questions, please leave a comments at below.

Reference

https://needone.app/python-importing-explained/