Zen

Source string Source string

494/10000 · 494
English Actions
Nevertheless, switching the JIT compilation off before the profiling is not "a must". It is rather a recommendation, and in some cases, for example in a production environment, you may need to keep JIT compilation on to see the full picture of all the memory allocations. In this case, the majority of the ``INTERNAL`` events are most probably caused by traces.
As for investigating the Lua code with the help of profiling reports, it is always code-dependent and there can't be hundred per cent definite recommendations in this regard. Nevertheless, you can see some of the things in the :ref:`Profiling report analysis example <profiler_analysis>` later.
Also, below is the :ref:`FAQ <profiler_faq>` section with the questions that most probably can arise while using profiler.
FAQ
In this section, some profiler-related points are discussed in a Q&A format.
**Question (Q)**: Is the profiler suitable for C allocations or allocations inside C code?
**Answer (A)**: The profiler reports only allocation events caused by the Lua allocator. All Lua-related allocations, like table or string creation are reported. But the profiler doesn't report allocations made by ``malloc()`` or other non-Lua allocators. You can use ``valgrind`` to debug them.
**Q**: Why are there so many ``INTERNAL`` allocations in my profiling report? What does it mean?
**A**: ``INTERNAL`` means that these allocations/reallocations/deallocations are related to the internal LuaJIT structures or are made on traces. Currently, the profiler doesn't verbosely report allocations of objects that are made during trace execution. Try :ref:`adding jit.off() <profiler_usage_internal_jitoff>` before the profiler start.
**Q**: Why are there some reallocations/deallocations without an ``Overrides`` section?
**A**: These objects can be created before the profiler starts. Adding ``collectgarbage()`` before the profiler's start enables collecting all previously allocated objects that are dead when the profiler starts.
**Q**: Why are some objects not collected during profiling? Is it a memory leak?
**A**: LuaJIT uses incremental Garbage Collector (GC). A GC cycle may not be finished at the moment the profiler stops. Add ``collectgarbage()`` before stopping the profiler to collect all the dead objects for sure.
**Q**: Can I profile not just a current chunk but the entire running application? Can I start the profiler when the application is already running?
**A**: Yes. Here is an example of code that can be inserted in the Tarantool console for a running instance.
local fiber = require "fiber"
local log = require "log"

fiber.create(function()
fiber.name("memprof")

collectgarbage() -- Collect all objects already dead
log.warn("start of profile")

local st, err = misc.memprof.start(FILENAME)
if not st then
log.error("failed to start profiler: %s", err)
end

fiber.sleep(TIME)

collectgarbage()
st, err = misc.memprof.stop()

if not st then
log.error("profiler on stop error: %s", err)
end

log.warn("end of profile")
end)
where
``FILENAME``—the name of the binary file where profiling events are written
``TIME``—duration of profiling, in seconds.
Also, you can directly call ``misc.memprof.start()`` and ``misc.memprof.stop()`` from a console.
Profiling report analysis example
In the example below, the following Lua code named ``format_concat.lua`` is investigated with the help of the memory profiler reports.
-- Prevent allocations on new traces.
jit.off()

local function concat(a)
local nstr = a.."a"
return nstr
end

local function format(a)
local nstr = string.format("%sa", a)
return nstr
end

collectgarbage()

local binfile = "/tmp/memprof_"..(arg[0]):match("([^/]*).lua")..".bin"

local st, err = misc.memprof.start(binfile)
assert(st, err)

-- Payload.
for i = 1, 10000 do
local f = format(i)
local c = concat(i)
end
collectgarbage()

local st, err = misc.memprof.stop()
assert(st, err)

os.exit()
When you run this code :ref:`under Tarantool <profiler_usage_generate>` and then :ref:`parse <profiler_usage_parse_command>` the binary memory profile in /tmp/memprof_format_concat.bin, you will get the following profiling report:
ALLOCATIONS
@format_concat.lua:10: 19996 events +624284 bytes -0 bytes
INTERNAL: 1 events +65536 bytes -0 bytes

REALLOCATIONS

DEALLOCATIONS
INTERNAL: 19996 events +0 bytes -558778 bytes
Overrides:
@format_concat.lua:10

@format_concat.lua:10: 2 events +0 bytes -98304 bytes
Overrides:
@format_concat.lua:10

HEAP SUMMARY:
INTERNAL holds 65536 bytes: 1 allocs, 0 frees
Reasonable questions regarding the report can be:
Why are there no allocations related to the ``concat()`` function?
Why is the number of allocations not a round number?
Why are there about 20K allocations instead of 10K?
First of all, LuaJIT doesn't create a new string if the string with the same payload exists (see details on `lua-users.org/wiki <http://lua-users.org/wiki/ImmutableObjects>`_). This is called `string interning <https://en.wikipedia.org/wiki/String_interning>`_. So, when a string is created via the ``format()`` function, there is no need to create the same string via the ``concat()`` function, and LuaJIT just uses the previous one.
That is also the reason why the number of allocations is not a round number as could be expected from the cycle operator ``for i = 1, 10000...``: Tarantool creates some strings for internal needs and built‑in modules, so some strings already exist.

Loading…

User avatar None

New source string

testing_project / Documentation: reference/tooling/luajit_memprof — English

local fiber = require "fiber"
local log = require "log"

fiber.create(function()
fiber.name("memprof")

collectgarbage() -- Collect all objects already dead
log.warn("start of profile")

local st, err = misc.memprof.start(FILENAME)
if not st then
log.error("failed to start profiler: %s", err)
end

fiber.sleep(TIME)

collectgarbage()
st, err = misc.memprof.stop()

if not st then
log.error("profiler on stop error: %s", err)
end

log.warn("end of profile")
end)
a year ago
Browse all string changes

Things to check

Multiple unnamed variables

There are multiple unnamed variables in the string, making it impossible for translators to reorder them

Reset

Glossary

English English
No related strings found in the glossary.

String information

Flags
python-format, read-only
String age
a year ago
Last updated
a year ago
Source string age
a year ago
Translation file
string 66