Added in version 3.4.
Source code:
———
The tracemalloc module is a debug tool to trace memory blocks allocated by Python. It provides the following information:
Traceback where an object was allocated
Statistics on allocated memory blocks per filename and per line number: total size, number and average size of allocated memory blocks
Compute the differences between two snapshots to detect memory leaks
To trace most memory blocks allocated by Python, the module should be started as early as possible by setting the
environment variable to 1, or by using
tracemalloc command line option. The
function can be called at runtime to start tracing Python memory allocations.
By default, a trace of an allocated memory block only stores the most recent frame (1 frame). To store 25 frames at startup: set the
environment variable to 25, or use the
tracemalloc=25 command line option.
Examples
Display the top 10
Display the 10 files allocating the most memory:
importtracemalloctracemalloc.start()# ... run your application ...snapshot=tracemalloc.take_snapshot()top_stats=snapshot.statistics('lineno')print("[ Top 10 ]")forstatintop_stats[:10]:print(stat)Example of output of the Python test suite:
[Top10]<frozenimportlib._bootstrap>:716:size=4855KiB,count=39328,average=126B<frozenimportlib._bootstrap>:284:size=521KiB,count=3199,average=167B/usr/lib/python3.4/collections/__init__.py:368:size=244KiB,count=2315,average=108B/usr/lib/python3.4/unittest/case.py:381:size=185KiB,count=779,average=243B/usr/lib/python3.4/unittest/case.py:402:size=154KiB,count=378,average=416B/usr/lib/python3.4/abc.py:133:size=88.7KiB,count=347,average=262B<frozenimportlib._bootstrap>:1446:size=70.4KiB,count=911,average=79B<frozenimportlib._bootstrap>:1454:size=52.0KiB,count=25,average=2131B<string>:5:size=49.7KiB,count=148,average=344B/usr/lib/python3.4/sysconfig.py:411:size=48.0KiB,count=1,average=48.0KiBWe can see that Python loaded 4855KiB data (bytecode and constants) from modules and that the
module allocated 244KiB to build
types.
See
for more options.
Compute differences
Take two snapshots and display the differences:
importtracemalloctracemalloc.start()# ... start your application ...snapshot1=tracemalloc.take_snapshot()# ... call the function leaking memory ...snapshot2=tracemalloc.take_snapshot()top_stats=snapshot2.compare_to(snapshot1,'lineno')print("[ Top 10 differences ]")forstatintop_stats[:10]:print(stat)Example of output before/after running some tests of the Python test suite:
[Top10differences]<frozenimportlib._bootstrap>:716:size=8173KiB(+4428KiB),count=71332(+39369),average=117B/usr/lib/python3.4/linecache.py:127:size=940KiB(+940KiB),count=8106(+8106),average=119B/usr/lib/python3.4/unittest/case.py:571:size=298KiB(+298KiB),count=589(+589),average=519B<frozenimportlib._bootstrap>:284:size=1005KiB(+166KiB),count=7423(+1526),average=139B/usr/lib/python3.4/mimetypes.py:217:size=112KiB(+112KiB),count=1334(+1334),average=86B/usr/lib/python3.4/http/server.py:848:size=96.0KiB(+96.0KiB),count=1(+1),average=96.0KiB/usr/lib/python3.4/inspect.py:1465:size=83.5KiB(+83.5KiB),count=109(+109),average=784B/usr/lib/python3.4/unittest/mock.py:491:size=77.7KiB(+77.7KiB),count=143(+143),average=557B/usr/lib/python3.4/urllib/parse.py:476:size=71.8KiB(+71.8KiB),count=969(+969),average=76B/usr/lib/python3.4/contextlib.py:38:size=67.2KiB(+67.2KiB),count=126(+126),average=546BWe can see that Python has loaded 8173KiB of module data (bytecode and constants), and that this is 4428KiB more than had been loaded before the tests, when the previous snapshot was taken. Similarly, the
module has cached 940KiB of Python source code to format tracebacks, all of it since the previous snapshot.
If the system has little free memory, snapshots can be written on disk using the
method to analyze the snapshot offline. Then use the
method reload the snapshot.
Get the traceback of a memory block
Code to display the traceback of the biggest memory block:
importtracemalloc# Store 25 framestracemalloc.start(25)# ... run your application ...snapshot=tracemalloc.take_snapshot()top_stats=snapshot.statistics('traceback')# pick the biggest memory blockstat=top_stats[0]print("%s memory blocks: %.1f KiB"%(stat.count,stat.size/1024))forlineinstat.traceback.format():print(line)Example of output of the Python test suite (traceback limited to 25 frames):
903memoryblocks:870.1KiBFile"<frozen importlib._bootstrap>",line716File"<frozen importlib._bootstrap>",line1036File"<frozen importlib._bootstrap>",line934File"<frozen importlib._bootstrap>",line1068File"<frozen importlib._bootstrap>",line619File"<frozen importlib._bootstrap>",line1581File"<frozen importlib._bootstrap>",line1614File"/usr/lib/python3.4/doctest.py",line101importpdbFile"<frozen importlib._bootstrap>",line284File"<frozen importlib._bootstrap>",line938File"<frozen importlib._bootstrap>",line1068File"<frozen importlib._bootstrap>",line619File"<frozen importlib._bootstrap>",line1581File"<frozen importlib._bootstrap>",line1614File"/usr/lib/python3.4/test/support/__init__.py",line1728importdoctestFile"/usr/lib/python3.4/test/test_pickletools.py",line21support.run_doctest(pickletools)File"/usr/lib/python3.4/test/regrtest.py",line1276test_runner()File"/usr/lib/python3.4/test/regrtest.py",line976display_failure=notverbose)File"/usr/lib/python3.4/test/regrtest.py",line761match_tests=ns.match_tests)File"/usr/lib/python3.4/test/regrtest.py",line1563main()File"/usr/lib/python3.4/test/__main__.py",line3regrtest.main_in_temp_cwd()File"/usr/lib/python3.4/runpy.py",line73exec(code,run_globals)File"/usr/lib/python3.4/runpy.py",line160"__main__",fname,loader,pkg_name)We can see that the most memory was allocated in the
module to load data (bytecode and constants) from modules: 870.1KiB. The traceback is where the importlib loaded data most recently: on the importpdb line of the
module. The traceback may change if a new module is loaded.
Pretty top
Code to display the 10 lines allocating the most memory with a pretty output, ignoring <frozenimportlib._bootstrap> and <unknown> files:
importlinecacheimportosimporttracemallocdefdisplay_top(snapshot,key_type='lineno',limit=10):snapshot=snapshot.filter_traces((tracemalloc.Filter(False,"<frozen importlib._bootstrap>"),tracemalloc.Filter(False,"<unknown>"),))top_stats=snapshot.statistics(key_type)print("Top %s lines"%limit)forindex,statinenumerate(top_stats[:limit],1):frame=stat.traceback[0]print("#%s: %s:%s: %.1f KiB"%(index,frame.filename,frame.lineno,stat.size/1024))line=linecache.getline(frame.filename,frame.lineno).strip()ifline:print(' %s'%line)other=top_stats[limit:]ifother:size=sum(stat.sizeforstatinother)print("%s other: %.1f KiB"%(len(other),size/1024))total=sum(stat.sizeforstatintop_stats)print("Total allocated size: %.1f KiB"%(total/1024))tracemalloc.start()# ... run your application ...snapshot=tracemalloc.take_snapshot()display_top(snapshot)Example of output of the Python test suite:
Top10lines#1: Lib/base64.py:414: 419.8 KiB_b85chars2=[(a+b)forain_b85charsforbin_b85chars]#2: Lib/base64.py:306: 419.8 KiB_a85chars2=[(a+b)forain_a85charsforbin_a85chars]#3: collections/__init__.py:368: 293.6 KiBexec(class_definition,namespace)#4: Lib/abc.py:133: 115.2 KiBcls=super().__new__(mcls,name,bases,namespace)#5: unittest/case.py:574: 103.1 KiBtestMethod()#6: Lib/linecache.py:127: 95.4 KiBlines=fp.readlines()#7: urllib/parse.py:476: 71.8 KiBforain_hexdigforbin_hexdig}#8: <string>:5: 62.0 KiB#9: Lib/_weakrefset.py:37: 60.0 KiBself.data=set()#10: Lib/base64.py:142: 59.8 KiB_b32tab2=[a+bforain_b32tabforbin_b32tab]6220other:3602.8KiBTotalallocatedsize:5303.1KiBSee
for more options.
Record the current and peak size of all traced memory blocks
The following code computes two sums like 0+1+2+... inefficiently, by creating a list of those numbers. This list consumes a lot of memory temporarily. We can use
and
to observe the small memory usage after the sum is computed as well as the peak memory usage during the computations:
importtracemalloctracemalloc.start()# Example code: compute a sum with a large temporary listlarge_sum=sum(list(range(100000)))first_size,first_peak=tracemalloc.get_traced_memory()tracemalloc.reset_peak()# Example code: compute a sum with a small temporary listsmall_sum=sum(list(range(1000)))second_size,second_peak=tracemalloc.get_traced_memory()print(f"{first_size=}, {first_peak=}")print(f"{second_size=}, {second_peak=}")Output:
first_size=664,first_peak=3592984second_size=804,second_peak=29704Using
ensured we could accurately record the peak during the computation of small_sum, even though it is much smaller than the overall peak size of memory blocks since the
call. Without the call to reset_peak(), second_peak would still be the peak from the computation large_sum (that is, equal to first_peak). In this case, both peaks are much higher than the final memory usage, and which suggests we could optimise (by removing the unnecessary call to
, and writing sum(range(...))).
API
Functions
tracemalloc.clear_traces()
Clear traces of memory blocks allocated by Python.
See also
.
tracemalloc.get_object_traceback(obj)
Get the traceback where the Python object obj was allocated. Return a
instance, or None if the tracemalloc module is not tracing memory allocations or did not trace the allocation of the object.
See also
and
functions.
tracemalloc.get_traceback_limit()
Get the maximum number of frames stored in the traceback of a trace.
The tracemalloc module must be tracing memory allocations to get the limit, otherwise an exception is raised.
The limit is set by the
function.
tracemalloc.get_traced_memory()
Get the current size and peak size of memory blocks traced by the tracemalloc module as a tuple: (current:int,peak:int).
tracemalloc.reset_peak()
Set the peak size of memory blocks traced by the tracemalloc module to the current size.
Do nothing if the tracemalloc module is not tracing memory allocations.
This function only modifies the recorded peak size, and does not modify or clear any traces, unlike
. Snapshots taken with
before a call to reset_peak() can be meaningfully compared to snapshots taken after the call.
See also
.
Added in version 3.9.
tracemalloc.get_tracemalloc_memory()
Get the memory usage in bytes of the tracemalloc module used to store traces of memory blocks. Return an
.
tracemalloc.is_tracing()
True if the tracemalloc module is tracing Python memory allocations, False otherwise.
See also
and
functions.
tracemalloc.start(nframe:
=1)
Start tracing Python memory allocations: install hooks on Python memory allocators. Collected tracebacks of traces will be limited to nframe frames. By default, a trace of a memory block only stores the most recent frame: the limit is 1. nframe must be greater or equal to 1.
You can still read the original number of total frames that composed the traceback by looking at the
attribute.
Storing more than 1 frame is only useful to compute statistics grouped by 'traceback' or to compute cumulative statistics: see the
and
methods.
Storing more frames increases the memory and CPU overhead of the tracemalloc module. Use the
function to measure how much memory is used by the tracemalloc module.
The
environment variable (PYTHONTRACEMALLOC=NFRAME) and the
tracemalloc=NFRAME command line option can be used to start tracing at startup.
See also
,
and
functions.
tracemalloc.stop()
Stop tracing Python memory allocations: uninstall hooks on Python memory allocators. Also clears all previously collected traces of memory blocks allocated by Python.
Call
function to take a snapshot of traces before clearing them.
See also
,
and
functions.
tracemalloc.take_snapshot()
Take a snapshot of traces of memory blocks allocated by Python. Return a new
instance.
The snapshot does not include memory blocks allocated before the tracemalloc module started to trace memory allocations.
Tracebacks of traces are limited to
frames. Use the nframe parameter of the
function to store more frames.
The tracemalloc module must be tracing memory allocations to take a snapshot, see the
function.
See also the
function.
DomainFilter
classtracemalloc.DomainFilter(inclusive:
, domain:
)
Filter traces of memory blocks by their address space (domain).
Added in version 3.6.
inclusive
If inclusive is True (include), match memory blocks allocated in the address space
.
If inclusive is False (exclude), match memory blocks not allocated in the address space
.
domain
Address space of a memory block (int). Read-only property.
Filter
classtracemalloc.Filter(inclusive:
, filename_pattern:
, lineno:
=None, all_frames:
=False, domain:
=None)
Filter on traces of memory blocks.
See the
function for the syntax of filename_pattern. The '.pyc' file extension is replaced with '.py'.
Examples:
Filter(True,subprocess.__file__) only includes traces of the
module
Filter(False,tracemalloc.__file__) excludes traces of the tracemalloc module
Filter(False,"<unknown>") excludes empty tracebacks
Changed in version 3.5: The '.pyo' file extension is no longer replaced with '.py'.
Changed in version 3.6: Added the
attribute.
domain
Address space of a memory block (int or None).
tracemalloc uses the domain 0 to trace memory allocations made by Python. C extensions can use other domains to trace other resources.
inclusive
If inclusive is True (include), only match memory blocks allocated in a file with a name matching
at line number
.
If inclusive is False (exclude), ignore memory blocks allocated in a file with a name matching
at line number
.
lineno
Line number (int) of the filter. If lineno is None, the filter matches any line number.
filename_pattern
Filename pattern of the filter (str). Read-only property.
all_frames
If all_frames is True, all frames of the traceback are checked. If all_frames is False, only the most recent frame is checked.
This attribute has no effect if the traceback limit is 1. See the
function and
attribute.
Frame
classtracemalloc.Frame
Frame of a traceback.
The
class is a sequence of Frame instances.
filename
Filename (str).
lineno
Line number (int).
Snapshot
classtracemalloc.Snapshot
Snapshot of traces of memory blocks allocated by Python.
The
function creates a snapshot instance.
compare_to(old_snapshot:Snapshot, key_type:
, cumulative:
=False)
Compute the differences with an old snapshot. Get statistics as a sorted list of
instances grouped by key_type.
See the
method for key_type and cumulative parameters.
The result is sorted from the biggest to the smallest by: absolute value of
,
, absolute value of
,
and then by
.
dump(filename)
Write the snapshot into a file.
Use
to reload the snapshot.
filter_traces(filters)
Create a new Snapshot instance with a filtered
sequence, filters is a list of
and
instances. If filters is an empty list, return a new Snapshot instance with a copy of the traces.
All inclusive filters are applied at once, a trace is ignored if no inclusive filters match it. A trace is ignored if at least one exclusive filter matches it.
Changed in version 3.6:
instances are now also accepted in filters.
classmethodload(filename)
Load a snapshot from a file.
See also
.
statistics(key_type:
, cumulative:
=False)
Get statistics as a sorted list of
instances grouped by key_type:
key_type
description
'filename'
filename
'lineno'
filename and line number
'traceback'
traceback
If cumulative is True, cumulate size and count of memory blocks of all frames of the traceback of a trace, not only the most recent frame. The cumulative mode can only be used with key_type equal to 'filename' and 'lineno'.
The result is sorted from the biggest to the smallest by:
,
and then by
.
traceback_limit
Maximum number of frames stored in the traceback of
: result of the
when the snapshot was taken.
traces
Traces of all memory blocks allocated by Python: sequence of
instances.
The sequence has an undefined order. Use the
method to get a sorted list of statistics.
Statistic
classtracemalloc.Statistic
Statistic on memory allocations.
returns a list of Statistic instances.
See also the
class.
count
Number of memory blocks (int).
size
Total size of memory blocks in bytes (int).
traceback
Traceback where the memory block was allocated,
instance.
StatisticDiff
classtracemalloc.StatisticDiff
Statistic difference on memory allocations between an old and a new
instance.
returns a list of StatisticDiff instances. See also the
class.
count
Number of memory blocks in the new snapshot (int): 0 if the memory blocks have been released in the new snapshot.
count_diff
Difference of number of memory blocks between the old and the new snapshots (int): 0 if the memory blocks have been allocated in the new snapshot.
size
Total size of memory blocks in bytes in the new snapshot (int): 0 if the memory blocks have been released in the new snapshot.
size_diff
Difference of total size of memory blocks in bytes between the old and the new snapshots (int): 0 if the memory blocks have been allocated in the new snapshot.
traceback
Traceback where the memory blocks were allocated,
instance.
Trace
classtracemalloc.Trace
Trace of a memory block.
The
attribute is a sequence of Trace instances.
Changed in version 3.6: Added the
attribute.
domain
Address space of a memory block (int). Read-only property.
tracemalloc uses the domain 0 to trace memory allocations made by Python. C extensions can use other domains to trace other resources.
size
Size of the memory block in bytes (int).
traceback
Traceback where the memory block was allocated,
instance.
Traceback
classtracemalloc.Traceback
Sequence of
instances sorted from the oldest frame to the most recent frame.
A traceback contains at least 1 frame. If the tracemalloc module failed to get a frame, the filename "<unknown>" at line number 0 is used.
When a snapshot is taken, tracebacks of traces are limited to
frames. See the
function. The original number of frames of the traceback is stored in the
attribute. That allows one to know if a traceback has been truncated by the traceback limit.
The
attribute is a Traceback instance.
Changed in version 3.7: Frames are now sorted from the oldest to the most recent, instead of most recent to oldest.
total_nframe
Total number of frames that composed the traceback before truncation. This attribute can be set to None if the information is not available.
Changed in version 3.9: The
attribute was added.
format(limit=None, most_recent_first=False)
Format the traceback as a list of lines. Use the
module to retrieve lines from the source code. If limit is set, format the limit most recent frames if limit is positive. Otherwise, format the abs(limit) oldest frames. If most_recent_first is True, the order of the formatted frames is reversed, returning the most recent frame first instead of last.
Similar to the
function, except that format() does not include newlines.
Example:
print("Traceback (most recent call first):")forlineintraceback:print(line)Output:
Traceback(mostrecentcallfirst):File"test.py",line9obj=Object()File"test.py",line12tb=tracemalloc.get_object_traceback(f())