When I want to import a library in Python, there are different methods. I always opt for the most common ones, but when I need to put the code into production, to make it go faster I always import the functions directly. For example:
Imports when I'm in development
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
from pygame.locals import *
Imports in production
from pandas import read_csv, DataFrame, concat
from numpy import array, zeros
I do this for two things:
- When I execute the functions, I understand that the call is faster when calling the function
- Importing a function at startup is faster than importing the entire package
Does this really make my code more optimal?
Quick answer: NO
This only makes the code less readable and more difficult to maintain both for you and for the other programmers who use it. In addition to the fact that you may have libraries with the same functions and one overwrites the other, causing problems. What you should do is follow the conventions of the language and of each library.
We are going to go into detail, for this we must know how Python performs imports.
How does Python import?
Summarizing when Python imports the libraries it does the following operations:
sys.modules
sys.path
and looks for the module, if it doesn't find it it will give aModuleNotFoundError
, if it finds it it goes to the next operation. (This is the most complex part of the import, but to focus on the question we are going to summarize it like this)types.ModuleType
.sys.module
with the name as the key and where the module is located (adds it to the cache).globals()
to be able to use it.It is important to note that when the module is imported for the first time, all the module code is executed and this is only done once and is loaded into memory , not every time the function is executed.
Python, unlike other languages, performs imports when the code is launched (at run time). This is different from, for example, C, C++... where modules are compiled and linked at compile time.
With this it is already deduced that there is no optimization of time or space , since even if you import a function, Python will always compile and execute the module in the same way and it is loaded. But let's see then how the different ways of importing change.
Ways to import
We are going to summarize all of the above as the loading process, so the following would happen in the different ways:
Importing the module
import math
math
into memory if it was not previously loaded.sys.modules
with the keymath
math
ournamespace
referencing object so we can call any function from this library withmath
sys.modules
math.
Creating an alias for the module
import math as py_math
math
into memory if it was not previously loaded.sys.modules
with the keymath
py_math
ournamespace
referencing object so we can call any function from this library withmath
sys.modules
py_math.
Importing a function from a module
import math import sin
(this is the most controversial when it comes to believing that it is more efficient)math
into memory if it was not previously loaded.sys.modules
with the keymath
sin
to ournamespace
referencing the objectmath.sin
,sys.modules
so when we runsin
python it will callmath.sin
math
in ournamespace
, so we will not be able to use the rest of its functions.Importing all module functions
from math import *
math
into memory if it was not previously loaded.sys.modules
with the keymath
.math
(and that allow us) to ournamespace
.math
in ournamespace
.As we can see, Python always performs the same operations except for how it puts the reference(s) in our
namespace
. In other words , the only thing that changes is how you call that package, or how you call the function. (this entails, as we see later, time changes, but they are ridiculous and unimportant) Nothing more.In no case are you "partially importing the package" or "importing the package every time you call the function" or anything like that.
Comparison of times
We are going to see the comparison of times, both when importing modules and when executing functions.
Import of modules
Let's check how all of the above is true, (or partially true) by executing code. For this we are going to create different imports and we are going to execute them 10M times, 5 repetitions:
Departure:
Importing a module with aliases or without aliases is exactly the same. But contrary to what our intuition might tell us, importing a function directly is 5 times slower than importing the entire module.
This happens because the import of the entire module, what it does is directly take the entire module and put it in our
namespace
, while the import of a function needs to load the entire module as before, but then it has to look for the function inside the module , to bring into our namespace just the function. This extra step is what creates the time increase.Use of functions
We are now going to see how the use of functions will be, if it is faster to perform
math.sin(2)
or import said function first and then executesin(2)
Departure:
Using the function directly
sin(2)
is 30% faster than using themath.sin(2)
. This makes sense, since the opposite happens to us than in the import.By having the function in it,
namespace
it only has to take it and execute it, while when we call it to the module, Python has the module in the namespace and inside it has to look for the functionsin()
Conclusions
Arrived here you may have made a mess. First I have said that everything was the same, and then I have shown that it is not the same. The really heavy thing, which is to load the module in memory, it always does the same thing, whatever you do it will always load the entire module in memory (no partial imports, or compile only one function). The only thing that adds or takes away time is the difference in looking up that Python does internally via dictionaries, and Python dictionaries are very, very fast.
Think about the tests, 10M imports? Who makes 10M imports? Using a function 10M times to get performance under 30 hundredths of a second is ridiculous.
The conclusion to be drawn is that Python has a uniform and really optimized way of working with modules.
What should be done, as I indicated at the beginning, is to follow the conventions that exist in the language and in the different libraries , because if it is not done, the only thing that is achieved is that the code is less readable, more difficult to maintain and understand for yourself, and for other programmers.