Skip to content

Add lazy module import caching API#47

Merged
benoitc merged 4 commits intomainfrom
feature/import-caching
Mar 22, 2026
Merged

Add lazy module import caching API#47
benoitc merged 4 commits intomainfrom
feature/import-caching

Conversation

@benoitc
Copy link
Owner

@benoitc benoitc commented Mar 22, 2026

Summary

  • Add explicit import caching for Python modules and functions
  • py:import/1 - Import and cache a module
  • py:import/2 - Import module and cache a specific function
  • py:flush_imports/0 - Clear all import caches
  • py:import_stats/0 - Get cache entry count
  • py:import_list/0 - List cached imports as #{Module => [Funcs]}

Caching is per-process (via PID affinity) and __main__ is never cached.

benoitc added 4 commits March 22, 2026 17:07
Adds explicit import caching for Python modules and functions:

API:
- py:import/1 - Import and cache a module
- py:import/2 - Import module and cache a specific function
- py:flush_imports/0 - Clear all import caches
- py:import_stats/0 - Get cache entry count
- py:import_list/0 - List cached imports as #{Module => [Funcs]}

Key behaviors:
- __main__ is never cached (returns error)
- Caching is per-process namespace (via PID affinity)
- All Python execution commands benefit from the cache

Tests:
- py_import_SUITE: 12 tests for basic import functionality
- py_import_owngil_SUITE: 9 tests for OWN_GIL mode (Python 3.12+)
- Multi-process and concurrent stress tests included
When using py_context:exec to define a function and then py_context:call
with '__main__' as the module, the function was not found because
'__main__' was looked up via PyImport_ImportModule (real Python module)
instead of ctx->globals where exec stores defined functions.

Add special handling to nif_context_call and owngil_execute_call to check
ctx->globals first when module is '__main__', matching the existing
behavior in nif_context_call_with_env.
string.ascii_lowercase is a string constant, not a callable function.
The test was failing for contexts that included the string module because
calling it raised TypeError. Use string.capwords instead.
- new/1: Create context with options map (mode => auto|subinterp|worker|owngil)
- destroy/1: Alias for stop/1 for API consistency
- call/4: Wrapper for call/5 with empty kwargs
- eval/2: Wrapper for eval/3 with empty locals
@benoitc benoitc merged commit 4ac10a4 into main Mar 22, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant