Compare commits

...

No commits in common. 'c9' and 'cs10' have entirely different histories.
c9 ... cs10

@ -79,8 +79,21 @@ using the `-R` flag:
%generate_buildrequires
%pyproject_buildrequires -R
Alternatively, the runtime dependencies can be obtained by building the wheel and reading the metadata from the built wheel.
This can be enabled by using the `-w` flag.
Alternatively, if the project specifies its dependencies in the pyproject.toml
`[project]` table (as defined in [PEP 621](https://www.python.org/dev/peps/pep-0621/)),
the runtime dependencies can be obtained by reading that metadata.
This can be enabled by using the `-p` flag.
This flag supports reading both the runtime dependencies, and the selected extras
(see the `-x` flag described below).
Please note that not all build backends which use pyproject.toml support the
`[project]` table scheme.
For example, poetry-core (at least in 1.9.0) defines package metadata in the
custom `[tool.poetry]` table which is not supported by the `%pyproject_buildrequires` macro.
Finally, the runtime dependencies can be obtained by building the wheel and reading the metadata from the built wheel.
This can be enabled with the `-w` flag and cannot be combined with `-p`.
Support for building wheels with `%pyproject_buildrequires -w` is **provisional** and the behavior might change.
Please subscribe to Fedora's [python-devel list] if you use the option.
@ -111,6 +124,14 @@ For example, if upstream suggests installing test dependencies with
%generate_buildrequires
%pyproject_buildrequires -x testing
For projects that specify test requirements using [PEP 735] dependency groups,
these can be added using the `-g` flag.
Multiple groups can be supplied by repeating the flag or as a comma separated list.
For example, if upstream uses a dependency group called `tests`, the test deps would be generated by:
%generate_buildrequires
%pyproject_buildrequires -g tests
For projects that specify test requirements in their [tox] configuration,
these can be added using the `-t` flag (default tox environment)
or the `-e` flag followed by the tox environment.
@ -134,20 +155,26 @@ The `-e` option redefines `%{toxenv}` for further reuse.
Use `%{default_toxenv}` to get the default value.
The `-t`/`-e` option uses [tox-current-env]'s `--print-deps-to-file` behind the scenes.
It generates dependencies listed directly in `deps`,
dependencies defined through `extras`,
and on tox 4.22+ also dependencies defined through `dependency_groups`.
If your package specifies some tox plugins in `tox.requires`,
such plugins will be BuildRequired as well.
Not all plugins are guaranteed to play well with [tox-current-env],
in worst case, patch/sed the requirement out from the tox configuration.
Note that neither `-x` or `-t` can be used with `-R`,
Note that neither `-x` or `-t` can be used with `-R` or `-N`,
because runtime dependencies are always required for testing.
You can only use those options if the build backend supports the [prepare-metadata-for-build-wheel hook],
or together with `-w`.
or together with `-p` or `-w`.
However, using `-g` with `-R` or `-N` is supported because dependency groups don't need to be used for testing
and can be obtained by reading `pyproject.toml` only.
[tox]: https://tox.readthedocs.io/
[tox-current-env]: https://github.com/fedora-python/tox-current-env/
[prepare-metadata-for-build-wheel hook]: https://www.python.org/dev/peps/pep-0517/#prepare-metadata-for-build-wheel
[python-devel list]: https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org/
Additionally to generated requirements you can supply multiple file names to `%pyproject_buildrequires` macro.
Dependencies will be loaded from them:
@ -157,7 +184,7 @@ Dependencies will be loaded from them:
For packages not using build system you can use `-N` to entirely skip automatical
generation of requirements and install requirements only from manually specified files.
`-N` option implies `-R` and cannot be used in combination with other options mentioned above
(`-w`, `-e`, `-t`, `-x`).
(`-w`, `-e`, `-t`, `-x`, `-p`).
The `%pyproject_buildrequires` macro also accepts the `-r` flag for backward compatibility;
it means "include runtime dependencies" which has been the default since version 0-53.
@ -287,7 +314,13 @@ However, in Fedora packages, always list executables explicitly to avoid uninten
`%pyproject_save_files` can automatically mark license files with `%license` macro
and language (`*.mo`) files with `%lang` macro and appropriate language code.
Only license files declared via [PEP 639] `License-File` field are detected.
[PEP 639] is still a draft and can be changed in the future.
[PEP 639] is still provisional and can be changed in the future.
It is possible to use the `-l` flag to declare that a missing license should
terminate the build or `-L` (the default) to explicitly disable this check.
Packagers are encouraged to use the `-l` flag when the `%license` file is not manually listed in `%files`
to avoid accidentally losing the file in a future version.
When the `%license` file is manually listed in `%files`,
packagers can use the `-L` flag to ensure future compatibility in case the `-l` behavior eventually becomes a default.
Note that `%pyproject_save_files` uses data from the [RECORD file](https://www.python.org/dev/peps/pep-0627/).
If you wish to rename, remove or otherwise change the installed files of a package
@ -342,6 +375,12 @@ The `%pyproject_check_import` macro also accepts positional arguments with
additional qualified module names to check, useful for example if some modules are installed manually.
Note that filtering by `-t`/`-e` also applies to the positional arguments.
Another macro, `%_pyproject_check_import_allow_no_modules` allows to pass the import check,
even if no Python modules are detected in the package.
This may be a valid case for packages containing e.g. typing stubs.
Don't use this macro in Fedora packages.
It's only intended to be used in automated build environments such as Copr.
Generating Extras subpackages
-----------------------------
@ -373,6 +412,78 @@ These arguments are still required:
Multiple subpackages are generated when multiple names are provided.
Provisional: Declarative Buildsystem (RPM 4.20+)
------------------------------------------------
It is possible to reduce some of the spec boilerplate by using the provided
pyproject [declarative buildsystem].
This option is only available with RPM 4.20+ (e.g. in Fedora 41+).
The declarative buildsystem is **provisional** and the behavior might change.
Please subscribe to Fedora's [python-devel list] if you use the feature.
To enable the pyproject declarative buildsystem, use the following:
BuildSystem: pyproject
BuildOption(install): <options for %%pyproject_save_files>
That way, RPM will automatically fill-in the `%prep`, `%generate_buildrequires`,
`%build`, `%install`, and `%check` sections the following defaults:
%prep
%autosetup -p1 -C
%generate_buildrequires
%pyproject_buildrequires
%build
%pyproject_wheel
%install
%pyproject_install
%pyproject_save_files <options from BuildOption(install)>
%check
%pyproject_check_import
To pass options to the individual macros, use `BuildOption` (see the [documentation of declarative buildsystems][declarative buildsystem]).
# pass options for %%pyproject_save_files (mandatory when not overriding %%install)
BuildOption(install): -l _module +auto
# replace the default options for %%autosetup
BuildOption(prep): -S git_am -C
# pass options to %%pyproject_buildrequires
BuildOption(generate_buildrequires): docs-requirements.txt -t
# pass options to %%pyproject_wheel
BuildOption(build): -C--global-option=--no-cython-compile
# pass options to %%pyproject_check_import
BuildOption(check): -e '*.test*'
Alternatively, you can supply your own sections to override the automatic ones:
BuildOption(generate_buildrequires): -w
...
%build
# do nothing, the wheel was built in %%generate_buildrequires
You can append to end of the automatic sections:
%check -a
# run %%pytest after %%pyproject_check_import
%pytest
Or prepend to the beginning of them:
%prep -p
# run %%gpgverify before %%autosetup
%gpgverify -k2 -s1 -d0
[declarative buildsystem]: https://rpm-software-management.github.io/rpm/manual/buildsystem.html
Limitations
-----------
@ -422,6 +533,7 @@ so be prepared for problems.
[PEP 517]: https://www.python.org/dev/peps/pep-0517/
[PEP 518]: https://www.python.org/dev/peps/pep-0518/
[PEP 639]: https://www.python.org/dev/peps/pep-0639/
[PEP 735]: https://www.python.org/dev/peps/pep-0735/
[pip's documentation]: https://pip.pypa.io/en/stable/cli/pip_install/#vcs-support

@ -4,4 +4,14 @@
# this macro will cause the package with the real macro to be installed.
# When macros.pyproject is installed, it overrides this macro.
# Note: This needs to maintain the same set of options as the real macro.
%pyproject_buildrequires(rRxtNwe:C:) echo 'pyproject-rpm-macros' && exit 0
%pyproject_buildrequires(rRxtNwpe:g:C:) echo 'pyproject-rpm-macros' && exit 0
# Declarative buildsystem, requires RPM 4.20+ to work
# https://rpm-software-management.github.io/rpm/manual/buildsystem.html
# This is the minimal implementation to be in the srpm package,
# as required even before the BuildRequires are installed
%buildsystem_pyproject_conf() %nil
%buildsystem_pyproject_generate_buildrequires() %pyproject_buildrequires %*
%buildsystem_pyproject_build() %nil
%buildsystem_pyproject_install() %nil

@ -1,5 +1,9 @@
# This is a backward-compatible suffix used in all pyproject-rpm-macros directories
# For the main Python it's empty, for all others it's "-3.X"
%_pyproject_files_pkgversion %{expr:"%{python3_pkgversion}" != "3" ? "-%{python3_pkgversion}" : ""}
# This is a directory where wheels are stored and installed from, absolute
%_pyproject_wheeldir %{_builddir}%{?buildsubdir:/%{buildsubdir}}/pyproject-wheeldir
%_pyproject_wheeldir %{_builddir}%{?buildsubdir:/%{buildsubdir}}/pyproject-wheeldir%{_pyproject_files_pkgversion}
# This is a directory used as TMPDIR, where pip copies sources to and builds from, relative to PWD
# For proper debugsource packages, we create TMPDIR within PWD
@ -8,12 +12,12 @@
# This will be used in debugsource package paths (applies to extension modules only)
# NB: pytest collects tests from here if not hidden
# https://docs.pytest.org/en/latest/reference.html#confval-norecursedirs
%_pyproject_builddir %{_builddir}%{?buildsubdir:/%{buildsubdir}}/.pyproject-builddir
%_pyproject_builddir %{_builddir}%{?buildsubdir:/%{buildsubdir}}/.pyproject-builddir%{_pyproject_files_pkgversion}
# We prefix all created files with this value to make them unique
# Ideally, we would put them into %%{buildsubdir}, but that value changes during the spec
# The used value is similar to the one used to define the default %%buildroot
%_pyproject_files_prefix %{name}-%{version}-%{release}.%{_arch}
%_pyproject_files_prefix %{name}-%{version}-%{release}.%{_arch}%{_pyproject_files_pkgversion}
%pyproject_files %{_builddir}/%{_pyproject_files_prefix}-pyproject-files
%_pyproject_modules %{_builddir}/%{_pyproject_files_prefix}-pyproject-modules
@ -21,6 +25,11 @@
%_pyproject_record %{_builddir}/%{_pyproject_files_prefix}-pyproject-record
%_pyproject_buildrequires %{_builddir}/%{_pyproject_files_prefix}-pyproject-buildrequires
# Internal macro, takes %%set_build_flags and strips all the exports
# TODO: Make such a list an actual source of %%set_build_flags (in redhat-rpm-config)
# Cannot use %%gsub directly to preserve EL 9 compatibility
%_pyproject_build_flags %{lua:local exports = rpm.expand('%{set_build_flags} ;'); print((exports:gsub('%s*;+%s+export%s+[%u_]+%s*;+%s*', ' ')))}
# Avoid leaking %%{_pyproject_builddir} to pytest collection
# https://bugzilla.redhat.com/show_bug.cgi?id=1935212
# The value is read and used by the %%pytest and %%tox macros:
@ -29,7 +38,8 @@
%pyproject_wheel(C:) %{expand:\\\
%_set_pytest_addopts
mkdir -p "%{_pyproject_builddir}"
CFLAGS="${CFLAGS:-${RPM_OPT_FLAGS}}" LDFLAGS="${LDFLAGS:-${RPM_LD_FLAGS}}" TMPDIR="%{_pyproject_builddir}" \\\
%{_pyproject_build_flags} \\\
TMPDIR="%{_pyproject_builddir}" \\\
%{__python3} -Bs %{_rpmconfigdir}/redhat/pyproject_wheel.py %{?**} %{_pyproject_wheeldir}
}
@ -65,6 +75,10 @@ echo $(IFS=:; echo "${pyproject_build_lib[*]}")
%pyproject_install() %{expand:\\\
specifier=$(ls %{_pyproject_wheeldir}/*.whl | xargs basename --multiple | sed -E 's/([^-]+)-([^-]+)-.+\\\.whl/\\\1==\\\2/')
if [ -z $specifier ]; then
echo 'ERROR: %%%%pyproject_install found no wheel in %%%%{_pyproject_wheeldir} %{_pyproject_wheeldir}' >&2
exit 1
fi
TMPDIR="%{_pyproject_builddir}" %{__python3} -m pip install --root %{buildroot} --prefix %{_prefix} --no-deps --disable-pip-version-check --progress-bar off --verbose --ignore-installed --no-warn-script-location --no-index --no-cache-dir --find-links %{_pyproject_wheeldir} $specifier
if [ -d %{buildroot}%{_bindir} ]; then
%py3_shebang_fix %{buildroot}%{_bindir}/*
@ -101,17 +115,15 @@ fi
# Note: the three times nested questionmarked -i -f -F pattern means: If none of those options was used -- in that case, we inject our own -f
%pyproject_extras_subpkg(n:i:f:F) %{expand:%{?python_extras_subpkg:%{python_extras_subpkg%{?!-i:%{?!-f:%{?!-F: -f %{_pyproject_ghost_distinfo}}}} %**}}}
%pyproject_extras_subpkg(n:i:f:FaA) %{expand:%{?python_extras_subpkg:%{python_extras_subpkg%{?!-i:%{?!-f:%{?!-F: -f %{_pyproject_ghost_distinfo}}}} %**}}}
# Escaping an actual percentage sign in path by 8 signs has been verified in RPM 4.16 and 4.17.
# See this thread http://lists.rpm.org/pipermail/rpm-list/2021-June/002048.html
# Since RPM 4.19, 2 signs are needed instead.
# On the CI, we build tests/escape_percentages.spec to verify the assumptions.
# We should check RPM version here instead of Fedora/RHEL, but it's hard;
# see https://github.com/rpm-software-management/rpm/issues/2523
%pyproject_save_files() %{expand:\\\
%{expr:0%{?fedora} >= 39 || 0%{?rhel} >= 10 ? "RPM_PERCENTAGES_COUNT=2" : "RPM_PERCENTAGES_COUNT=8" } \\
# Escaping shell-globs, percentage signs and spaces was reworked in RPM 4.19+
# https://github.com/rpm-software-management/rpm/issues/1749#issuecomment-1020420616
# Since we support both ways, we pass either 4.19 or 4.18 to the script, so it knows which one to use
# Rather than passing the actual version, we let RPM compare the versions, as it is easier done here than in Python
%pyproject_save_files(lL) %{expand:\\\
%{expr:v"0%{?rpmversion}" >= v"4.18.90" ? "RPM_FILES_ESCAPE=4.19" : "RPM_FILES_ESCAPE=4.18" } \\
%{__python3} %{_rpmconfigdir}/redhat/pyproject_save_files.py \\
--output-files "%{pyproject_files}" \\
--output-modules "%{_pyproject_modules}" \\
@ -121,7 +133,7 @@ fi
--python-version "%{python3_version}" \\
--pyproject-record "%{_pyproject_record}" \\
--prefix "%{_prefix}" \\
%{*}
%{**}
}
# -t - Process only top-level modules
@ -135,22 +147,38 @@ fi
}
%_pyproject_check_import_allow_no_modules(e:t) \
if [ -z "$(cat %{_pyproject_modules})" ]; then\
echo "No modules to check found, exiting check"\
else\
%pyproject_check_import %{?**}\
fi
%default_toxenv py%{python3_version_nodots}
%toxenv %{default_toxenv}
%_pyproject_tomlidep %["%{python3_pkgversion}" == "3"\
? "echo '(python%{python3_pkgversion}dist(tomli) if python%{python3_pkgversion}-devel < 3.11)'"\
: "%[v"%{python3_pkgversion}" < v"3.11"\
? "echo 'python%{python3_pkgversion}dist(tomli)'"\
: "true # will use tomllib, echo nothing"\
]"\
]
# Note: Keep the options in sync with this macro from macros.aaa-pyproject-srpm
%pyproject_buildrequires(rRxtNwe:C:) %{expand:\\\
%pyproject_buildrequires(rRxtNwpe:g:C:) %{expand:\\\
%_set_pytest_addopts
# The _auto_set_build_flags feature does not do this in %%generate_buildrequires section,
# but we want to get an environment consistent with %%build:
%{?_auto_set_build_flags:%set_build_flags}
# The default flags expect the package note file to exist
# see https://bugzilla.redhat.com/show_bug.cgi?id=2097535
%{?_package_note_flags:%_generate_package_note_file}
%{-R:
%{-r:%{error:The -R and -r options are mutually exclusive}}
%{-x:%{error:The -R and -x options are mutually exclusive}}
%{-e:%{error:The -R and -e options are mutually exclusive}}
%{-t:%{error:The -R and -t options are mutually exclusive}}
%{-w:%{error:The -R and -w options are mutually exclusive}}
%{-p:%{error:The -R and -p options are mutually exclusive}}
}
%{-N:
%{-r:%{error:The -N and -r options are mutually exclusive}}
@ -158,25 +186,25 @@ fi
%{-e:%{error:The -N and -e options are mutually exclusive}}
%{-t:%{error:The -N and -t options are mutually exclusive}}
%{-w:%{error:The -N and -w options are mutually exclusive}}
%{-p:%{error:The -N and -p options are mutually exclusive}}
%{-C:%{error:The -N and -C options are mutually exclusive}}
%{-g:if [ -f pyproject.toml ]; then
%_pyproject_tomlidep
fi}
}
%{-w:
%{-p:%{error:The -w and -p options are mutually exclusive}}
}
%{-e:%{expand:%global toxenv %(%{__python3} -s %{_rpmconfigdir}/redhat/pyproject_construct_toxenv.py %{?**})}}
echo 'pyproject-rpm-macros' # first stdout line matches the implementation in macros.aaa-pyproject-srpm
echo 'python%{python3_pkgversion}-devel'
echo 'python%{python3_pkgversion}dist(pip) >= 19'
echo 'python%{python3_pkgversion}dist(packaging)'
%{!-N:if [ -f pyproject.toml ]; then
%["%{python3_pkgversion}" == "3"
? "echo '(python%{python3_pkgversion}dist(tomli) if python%{python3_pkgversion}-devel < 3.11)'"
: "%[v"%{python3_pkgversion}" < v"3.11"
? "echo 'python%{python3_pkgversion}dist(tomli)'"
: "true # will use tomllib, echo nothing"
]"
]
%{!-N:echo 'python%{python3_pkgversion}dist(pip) >= 19'
if [ -f pyproject.toml ]; then
%_pyproject_tomlidep
elif [ -f setup.py ]; then
# Note: If the default requirements change, also change them in the script!
echo 'python%{python3_pkgversion}dist(setuptools) >= 40.8'
echo 'python%{python3_pkgversion}dist(wheel)'
else
echo 'ERROR: Neither pyproject.toml nor setup.py found, consider using %%%%pyproject_buildrequires -N <requirements-file> if this is not a Python package.' >&2
exit 1
@ -186,7 +214,8 @@ rm -rfv *.dist-info/ >&2
if [ -f %{__python3} ]; then
mkdir -p "%{_pyproject_builddir}"
echo -n > %{_pyproject_buildrequires}
CFLAGS="${CFLAGS:-${RPM_OPT_FLAGS}}" LDFLAGS="${LDFLAGS:-${RPM_LD_FLAGS}}" TMPDIR="%{_pyproject_builddir}" \\\
%{_pyproject_build_flags} \\\
TMPDIR="%{_pyproject_builddir}" \\\
RPM_TOXENV="%{toxenv}" HOSTNAME="rpmbuild" %{__python3} -Bs %{_rpmconfigdir}/redhat/pyproject_buildrequires.py %{?!_python_no_extras_requires:--generate-extras} --python3_pkgversion %{python3_pkgversion} --wheeldir %{_pyproject_wheeldir} --output %{_pyproject_buildrequires} %{?**} >&2
cat %{_pyproject_buildrequires}
fi
@ -204,3 +233,13 @@ PYTHONPATH="${PYTHONPATH:-%{buildroot}%{python3_sitearch}:%{buildroot}%{python3_
HOSTNAME="rpmbuild" \\
%{__python3} -m tox --current-env -q --recreate -e "%{-e:%{-e*}}%{!-e:%{toxenv}}" %{?*}
}
# Declarative buildsystem, requires RPM 4.20+ to work
# https://rpm-software-management.github.io/rpm/manual/buildsystem.html
%buildsystem_pyproject_conf() %nil
%buildsystem_pyproject_generate_buildrequires() %pyproject_buildrequires %*
%buildsystem_pyproject_build() %pyproject_wheel %*
%buildsystem_pyproject_install() %["%{shrink:%*}" == "" ? "%{error:BuildOption(install) is mandatory with pyproject BuildSystem.}" : "%pyproject_install \
%pyproject_save_files %*"]
%buildsystem_pyproject_check() %pyproject_check_import %*

@ -10,6 +10,7 @@ import subprocess
import re
import tempfile
import email.parser
import functools
import pathlib
import zipfile
@ -34,6 +35,7 @@ def print_err(*args, **kwargs):
try:
from packaging.markers import Marker
from packaging.requirements import Requirement, InvalidRequirement
from packaging.utils import canonicalize_name
except ImportError as e:
@ -99,15 +101,20 @@ class Requirements:
return True
return False
def add(self, requirement_str, *, package_name=None, source=None):
def add(self, requirement, *, package_name=None, source=None, extra=None):
"""Output a Python-style requirement string as RPM dep"""
requirement_str = str(requirement)
print_err(f'Handling {requirement_str} from {source}')
# requirements read initially from the metadata are strings
# further on we work with them as Requirement instances
if not isinstance(requirement, Requirement):
try:
requirement = Requirement(requirement_str)
requirement = Requirement(requirement)
except InvalidRequirement:
hint = guess_reason_for_invalid_requirement(requirement_str)
message = f'Requirement {requirement_str!r} from {source} is invalid.'
hint = guess_reason_for_invalid_requirement(requirement)
message = f'Requirement {requirement!r} from {source} is invalid.'
if hint:
message += f' Hint: {hint}'
raise ValueError(message)
@ -118,10 +125,17 @@ class Requirements:
)
name = canonicalize_name(requirement.name)
if extra is not None:
extra_str = f'extra == "{extra}"'
if requirement.marker is not None:
extra_str = f'({requirement.marker}) and {extra_str}'
requirement.marker = Marker(extra_str)
if (requirement.marker is not None and
not self.evaluate_all_environments(requirement)):
print_err(f'Ignoring alien requirement:', requirement_str)
self.ignored_alien_requirements.append(requirement_str)
self.ignored_alien_requirements.append(requirement)
return
# Handle self-referencing requirements
@ -215,7 +229,8 @@ def toml_load(opened_binary_file):
return tomllib.load(opened_binary_file)
def get_backend(requirements):
@functools.cache
def load_pyproject():
try:
f = open('pyproject.toml', 'rb')
except FileNotFoundError:
@ -223,6 +238,11 @@ def get_backend(requirements):
else:
with f:
pyproject_data = toml_load(f)
return pyproject_data
def get_backend(requirements):
pyproject_data = load_pyproject()
buildsystem_data = pyproject_data.get('build-system', {})
requirements.extend(
@ -248,7 +268,6 @@ def get_backend(requirements):
# with pyproject.toml without a specified build backend.
# If the default requirements change, also change them in the macro!
requirements.add('setuptools >= 40.8', source='default build backend')
requirements.add('wheel', source='default build backend')
requirements.check(source='build backend')
@ -302,7 +321,9 @@ def generate_run_requirements_hook(backend, requirements):
raise ValueError(
'The build backend cannot provide build metadata '
'(incl. runtime requirements) before build. '
'Use the provisional -w flag to build the wheel and parse the metadata from it, '
'If the dependencies are specified in the pyproject.toml [project] '
'table, you can use the -p flag to read them.'
'Alternatively, use the provisional -w flag to build the wheel and parse the metadata from it, '
'or use the -R flag not to generate runtime dependencies.'
)
dir_basename = prepare_metadata('.', config_settings=requirements.config_settings)
@ -328,6 +349,11 @@ def generate_run_requirements_wheel(backend, requirements, wheeldir):
# Reuse the wheel from the previous round of %pyproject_buildrequires (if it exists)
wheel = find_built_wheel(wheeldir)
if not wheel:
# pip is already echoed from the macro
# but we need to explicitly restart if has not yet been installed
# see https://bugzilla.redhat.com/2169855
requirements.add('pip >= 19', source='%pyproject_buildrequires -w')
requirements.check(source='%pyproject_buildrequires -w')
import pyproject_wheel
returncode = pyproject_wheel.build_wheel(
wheeldir=wheeldir,
@ -355,8 +381,35 @@ def generate_run_requirements_wheel(backend, requirements, wheeldir):
raise RuntimeError('Could not find *.dist-info/METADATA in built wheel.')
def generate_run_requirements(backend, requirements, *, build_wheel, wheeldir):
if build_wheel:
def generate_run_requirements_pyproject(requirements):
pyproject_data = load_pyproject()
if not (project_table := pyproject_data.get('project', {})):
raise ValueError('Could not find the [project] table in pyproject.toml.')
dynamic_fields = project_table.get('dynamic', [])
if 'dependencies' in dynamic_fields or 'optional-dependencies' in dynamic_fields:
raise ValueError('Could not read the dependencies or optional-dependencies '
'from the [project] table in pyproject.toml, as the field is dynamic.')
dependencies = project_table.get('dependencies', [])
name = project_table.get('name')
requirements.extend(dependencies,
package_name=name,
source=f'pyproject.toml generated metadata: [dependencies] ({name})')
optional_dependencies = project_table.get('optional-dependencies', {})
for extra, dependencies in optional_dependencies.items():
requirements.extend(dependencies,
package_name=name,
source=f'pyproject.toml generated metadata: [optional-dependencies] {extra} ({name})',
extra=extra)
def generate_run_requirements(backend, requirements, *, build_wheel, read_pyproject_dependencies, wheeldir):
if read_pyproject_dependencies:
generate_run_requirements_pyproject(requirements)
elif build_wheel:
generate_run_requirements_wheel(backend, requirements, wheeldir)
else:
generate_run_requirements_hook(backend, requirements)
@ -386,7 +439,7 @@ def generate_tox_requirements(toxenv, requirements):
provision_content = provision.read()
if provision_content and r.returncode != 0:
provision_requires = json.loads(provision_content)
if 'minversion' in provision_requires:
if provision_requires.get('minversion') is not None:
requirements.add(f'tox >= {provision_requires["minversion"]}',
source='tox provision (minversion)')
if 'requires' in provision_requires:
@ -406,6 +459,103 @@ def generate_tox_requirements(toxenv, requirements):
source=f'tox --print-deps-only: {toxenv}')
def tox_dependency_groups(toxenv):
# We call this command separately instead of folding it into the previous one
# becasue --print-dependency-groups-to only works with tox 4.22+ and tox-current-env 0.0.14+.
# We handle failure gracefully: upstreams using dependency_groups should require tox >= 4.22.
toxenv = ','.join(toxenv)
with tempfile.NamedTemporaryFile('r') as groups:
r = subprocess.run(
[sys.executable, '-m', 'tox',
'--print-dependency-groups-to', groups.name,
'-q', '-e', toxenv],
check=False,
encoding='utf-8',
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
)
if r.returncode == 0:
if r.stdout:
print_err(r.stdout, end='')
if output := groups.read().strip():
return output.splitlines()
return []
def generate_dependency_groups(requested_groups, requirements):
"""Adapted from https://peps.python.org/pep-0735/#reference-implementation (public domain)"""
from collections import defaultdict
def _normalize_name(name: str) -> str:
return re.sub(r"[-_.]+", "-", name).lower()
def _normalize_group_names(dependency_groups: dict) -> dict:
original_names = defaultdict(list)
normalized_groups = {}
for group_name, value in dependency_groups.items():
normed_group_name = _normalize_name(group_name)
original_names[normed_group_name].append(group_name)
normalized_groups[normed_group_name] = value
errors = []
for normed_name, names in original_names.items():
if len(names) > 1:
errors.append(f"{normed_name} ({', '.join(names)})")
if errors:
raise ValueError(f"Duplicate dependency group names: {', '.join(errors)}")
return normalized_groups
def _resolve_dependency_group(
dependency_groups: dict, group: str, past_groups: tuple[str, ...] = ()
) -> list[str]:
if group in past_groups:
raise ValueError(f"Cyclic dependency group include: {group} -> {past_groups}")
if group not in dependency_groups:
raise LookupError(f"Dependency group '{group}' not found")
raw_group = dependency_groups[group]
if not isinstance(raw_group, list):
raise ValueError(f"Dependency group '{group}' is not a list")
realized_group = []
for item in raw_group:
if isinstance(item, str):
realized_group.append(item)
elif isinstance(item, dict):
if tuple(item.keys()) != ("include-group",):
raise ValueError(f"Invalid dependency group item: {item}")
include_group = _normalize_name(next(iter(item.values())))
realized_group.extend(
_resolve_dependency_group(
dependency_groups, include_group, past_groups + (group,)
)
)
else:
raise ValueError(f"Invalid dependency group item: {item}")
return realized_group
def resolve(dependency_groups: dict, group: str) -> list[str]:
if not isinstance(dependency_groups, dict):
raise TypeError("Dependency Groups table is not a dict")
return _resolve_dependency_group(dependency_groups, _normalize_name(group))
pyproject_data = load_pyproject()
dependency_groups_raw = pyproject_data.get("dependency-groups", {})
dependency_groups = _normalize_group_names(dependency_groups_raw)
for group_names in requested_groups:
for group_name in group_names.split(","):
requirements.extend(
resolve(dependency_groups, group_name),
source=f"Dependency group {group_name}",
)
def python3dist(name, op=None, version=None, python3_pkgversion="3"):
prefix = f"python{python3_pkgversion}dist"
@ -418,9 +568,10 @@ def python3dist(name, op=None, version=None, python3_pkgversion="3"):
def generate_requires(
*, include_runtime=False, build_wheel=False, wheeldir=None, toxenv=None, extras=None,
*, include_runtime=False, build_wheel=False, wheeldir=None, toxenv=None, extras=None, dependency_groups=None,
get_installed_version=importlib.metadata.version, # for dep injection
generate_extras=False, python3_pkgversion="3", requirement_files=None, use_build_system=True,
read_pyproject_dependencies=False,
output, config_settings=None,
):
"""Generate the BuildRequires for the project in the current directory
@ -436,9 +587,10 @@ def generate_requires(
config_settings=config_settings,
)
dependency_groups = dependency_groups or []
try:
if (include_runtime or toxenv) and not use_build_system:
raise ValueError('-N option cannot be used in combination with -r, -e, -t, -x options')
if (include_runtime or toxenv or read_pyproject_dependencies) and not use_build_system:
raise ValueError('-N option cannot be used in combination with -r, -e, -t, -x, -p options')
if requirement_files:
for req_file in requirement_files:
requirements.extend(
@ -452,8 +604,12 @@ def generate_requires(
if toxenv:
include_runtime = True
generate_tox_requirements(toxenv, requirements)
dependency_groups.extend(tox_dependency_groups(toxenv))
if dependency_groups:
generate_dependency_groups(dependency_groups, requirements)
if include_runtime:
generate_run_requirements(backend, requirements, build_wheel=build_wheel, wheeldir=wheeldir)
generate_run_requirements(backend, requirements, build_wheel=build_wheel,
read_pyproject_dependencies=read_pyproject_dependencies, wheeldir=wheeldir)
except EndPass:
return
finally:
@ -480,7 +636,7 @@ def main(argv):
help=argparse.SUPPRESS,
)
parser.add_argument(
'-p', '--python3_pkgversion', metavar='PYTHON3_PKGVERSION',
'--python3_pkgversion', metavar='PYTHON3_PKGVERSION',
default="3", help=argparse.SUPPRESS,
)
parser.add_argument(
@ -495,6 +651,11 @@ def main(argv):
help='comma separated list of "extras" for runtime requirements '
'(e.g. -x testing,feature-x) (implies --runtime, can be repeated)',
)
parser.add_argument(
'-g', '--dependency-groups', metavar='GROUPS', action='append',
help='comma separated list of dependency groups (PEP 735) for requirements '
'(e.g. -g tests,docs) (can be repeated)',
)
parser.add_argument(
'-t', '--tox', action='store_true',
help=('generate test tequirements from tox environment '
@ -510,6 +671,11 @@ def main(argv):
help=('Generate run-time requirements by building the wheel '
'(useful for build backends without the prepare_metadata_for_build_wheel hook)'),
)
parser.add_argument(
'-p', '--read-pyproject-dependencies', action='store_true', default=False,
help=('Generate dependencies from [project] table of pyproject.toml '
'instead of calling prepare_metadata_for_build_wheel hook)'),
)
parser.add_argument(
'-R', '--no-runtime', action='store_false', dest='runtime',
help="Don't generate run-time requirements (implied by -N)",
@ -558,10 +724,12 @@ def main(argv):
wheeldir=args.wheeldir,
toxenv=args.toxenv,
extras=args.extras,
dependency_groups=args.dependency_groups,
generate_extras=args.generate_extras,
python3_pkgversion=args.python3_pkgversion,
requirement_files=args.requirement_files,
use_build_system=args.use_build_system,
read_pyproject_dependencies=args.read_pyproject_dependencies,
output=args.output,
config_settings=parse_config_settings_args(args.config_settings),
)

@ -23,7 +23,6 @@ Insufficient version of setuptools:
setup.py: |
expected: |
python3dist(setuptools) >= 40.8
python3dist(wheel)
result: 0
No pyproject.toml, empty setup.py:
@ -32,9 +31,11 @@ No pyproject.toml, empty setup.py:
wheel: 1
include_runtime: false
setup.py: |
expected: |
expected:
- | # setuptools 70+
python3dist(setuptools) >= 40.8
- | # setuptools < 70
python3dist(setuptools) >= 40.8
python3dist(wheel)
python3dist(wheel)
result: 0
@ -47,9 +48,11 @@ Default build system, empty setup.py:
pyproject.toml: |
# empty
setup.py: |
expected: |
expected:
- | # setuptools 70+
python3dist(setuptools) >= 40.8
- | # setuptools < 70
python3dist(setuptools) >= 40.8
python3dist(wheel)
python3dist(wheel)
result: 0
@ -212,9 +215,14 @@ Default build system, build dependencies in setup.py:
setup_requires=['foo', 'bar!=2', 'baz~=1.1.1'],
install_requires=['inst'],
)
expected: |
expected:
- | # setuptools 70+
python3dist(setuptools) >= 40.8
python3dist(foo)
(python3dist(bar) < 2 or python3dist(bar) > 2)
(python3dist(baz) >= 1.1.1 with python3dist(baz) < 1.2)
- | # setuptools < 70
python3dist(setuptools) >= 40.8
python3dist(wheel)
python3dist(wheel)
python3dist(foo)
(python3dist(bar) < 2 or python3dist(bar) > 2)
@ -234,9 +242,14 @@ Default build system, run dependencies in setup.py:
setup_requires=['pyyaml'], # nb. setuptools will try to install this
install_requires=['inst > 1', 'inst2 < 3'],
)
expected: |
expected:
- | # setuptools 70+
python3dist(setuptools) >= 40.8
python3dist(pyyaml)
python3dist(inst) > 1.0
python3dist(inst2) < 3~~
- | # setuptools < 70
python3dist(setuptools) >= 40.8
python3dist(wheel)
python3dist(wheel)
python3dist(pyyaml)
python3dist(inst) > 1.0
@ -287,9 +300,19 @@ Run dependencies with extras (not selected):
if __name__ == "__main__":
main()
expected: |
expected:
- | # setuptools 70+
python3dist(setuptools) >= 40.8
python3dist(setuptools) >= 40
python3dist(py) >= 1.5
python3dist(six) >= 1.10
python3dist(setuptools)
python3dist(attrs) >= 17.4
python3dist(atomicwrites) >= 1
python3dist(pluggy) >= 0.11
python3dist(more-itertools) >= 4
- | # setuptools < 70
python3dist(setuptools) >= 40.8
python3dist(wheel)
python3dist(wheel)
python3dist(setuptools) >= 40
python3dist(py) >= 1.5
@ -310,9 +333,23 @@ Run dependencies with extras (selected):
extras:
- testing
setup.py: *pytest_setup_py
expected: |
expected:
- | # setuptools 70+
python3dist(setuptools) >= 40.8
python3dist(setuptools) >= 40
python3dist(py) >= 1.5
python3dist(six) >= 1.10
python3dist(setuptools)
python3dist(attrs) >= 17.4
python3dist(atomicwrites) >= 1
python3dist(pluggy) >= 0.11
python3dist(more-itertools) >= 4
python3dist(argcomplete)
python3dist(hypothesis) >= 3.56
python3dist(nose)
python3dist(requests)
- | # setuptools < 70
python3dist(setuptools) >= 40.8
python3dist(wheel)
python3dist(wheel)
python3dist(setuptools) >= 40
python3dist(py) >= 1.5
@ -348,9 +385,17 @@ Run dependencies with multiple extras:
'cool-feature': ['dep4[FOO,BAR]'],
},
)
expected: |
expected:
- | # setuptools 70+
python3dist(setuptools) >= 40.8
python3dist(dep4)
python3dist(dep4[bar])
python3dist(dep4[foo])
python3dist(dep3)
python3dist(dep2)
python3dist(dep1)
- | # setuptools < 70
python3dist(setuptools) >= 40.8
python3dist(wheel)
python3dist(wheel)
python3dist(dep4)
python3dist(dep4[bar])
@ -365,16 +410,33 @@ Run dependencies with extras and build wheel option:
setuptools: 50
wheel: 1
pyyaml: 1
pip: 20
include_runtime: true
build_wheel: true
extras:
- testing
setup.py: *pytest_setup_py
expected: |
expected:
- | # setuptools 70+
python3dist(setuptools) >= 40.8
python3dist(setuptools) >= 40
python3dist(pip) >= 19
python3dist(py) >= 1.5
python3dist(six) >= 1.10
python3dist(setuptools)
python3dist(attrs) >= 17.4
python3dist(atomicwrites) >= 1
python3dist(pluggy) >= 0.11
python3dist(more-itertools) >= 4
python3dist(argcomplete)
python3dist(hypothesis) >= 3.56
python3dist(nose)
python3dist(requests)
- | # setuptools < 70
python3dist(setuptools) >= 40.8
python3dist(wheel)
python3dist(wheel)
python3dist(setuptools) >= 40
python3dist(pip) >= 19
python3dist(py) >= 1.5
python3dist(six) >= 1.10
python3dist(setuptools)
@ -414,17 +476,22 @@ tox dependencies:
commands =
true
expected:
- | # tox 3
- | # tox 3 with setuptools < 70
python3dist(setuptools) >= 40.8
python3dist(wheel)
python3dist(wheel)
python3dist(tox-current-env) >= 0.0.6
python3dist(toxdep1)
python3dist(toxdep2)
python3dist(inst)
- | # tox 4
- | # tox 4 with setuptools 70+
python3dist(setuptools) >= 40.8
python3dist(tox-current-env) >= 0.0.6
python3dist(tox)
python3dist(toxdep1)
python3dist(toxdep2)
python3dist(inst)
- | # tox 4 with setuptools < 70
python3dist(setuptools) >= 40.8
python3dist(wheel)
python3dist(wheel)
python3dist(tox-current-env) >= 0.0.6
python3dist(tox)
@ -466,10 +533,9 @@ tox extras:
commands =
true
expected:
- | # tox 3
- | # tox 3 with setuptools < 70
python3dist(setuptools) >= 40.8
python3dist(wheel)
python3dist(wheel)
python3dist(tox-current-env) >= 0.0.6
python3dist(toxdep)
python3dist(inst)
@ -480,9 +546,21 @@ tox extras:
python3dist(dep23)
python3dist(extra-dep)
python3dist(extra-dep[extra_dep])
- | # tox 4
- | # tox 4 with setuptools 70+
python3dist(setuptools) >= 40.8
python3dist(tox-current-env) >= 0.0.6
python3dist(tox)
python3dist(toxdep)
python3dist(inst)
python3dist(dep11) > 11.0
python3dist(dep12)
python3dist(dep21)
python3dist(dep22)
python3dist(dep23)
python3dist(extra-dep)
python3dist(extra-dep[extra_dep])
- | # tox 4 with setuptools < 70
python3dist(setuptools) >= 40.8
python3dist(wheel)
python3dist(wheel)
python3dist(tox-current-env) >= 0.0.6
python3dist(tox)
@ -523,17 +601,22 @@ tox provision unsatisfied:
toxdep1
toxdep2
expected:
- | # tox 3
- | # tox 3 with setuptools < 70
python3dist(setuptools) >= 40.8
python3dist(wheel)
python3dist(wheel)
python3dist(tox-current-env) >= 0.0.6
python3dist(tox) >= 3.999
python3dist(setuptools) > 40.0
python3dist(wheel) > 2.0
- | # tox 4
- | # tox 4 with setuptools 70+
python3dist(setuptools) >= 40.8
python3dist(tox-current-env) >= 0.0.6
python3dist(tox) >= 3.999
python3dist(setuptools) > 40.0
python3dist(wheel) > 2.0
python3dist(tox) >= 3.999
- | # tox 4 with setuptools < 70
python3dist(setuptools) >= 40.8
python3dist(wheel)
python3dist(wheel)
python3dist(tox-current-env) >= 0.0.6
python3dist(tox) >= 3.999
@ -567,19 +650,25 @@ tox provision satisfied:
toxdep1
toxdep2
expected:
- | # tox 3
- | # tox 3 with setuptools < 70
python3dist(setuptools) >= 40.8
python3dist(wheel)
python3dist(wheel)
python3dist(tox-current-env) >= 0.0.6
python3dist(tox) >= 3.5
python3dist(setuptools) > 40.0
python3dist(toxdep1)
python3dist(toxdep2)
python3dist(inst)
- | # tox 4
- | # tox 4 with setuptools 70+
python3dist(setuptools) >= 40.8
python3dist(tox-current-env) >= 0.0.6
python3dist(setuptools) > 40.0
python3dist(tox) >= 3.5
python3dist(toxdep1)
python3dist(toxdep2)
python3dist(inst)
- | # tox 4 with setuptools < 70
python3dist(setuptools) >= 40.8
python3dist(wheel)
python3dist(wheel)
python3dist(tox-current-env) >= 0.0.6
python3dist(setuptools) > 40.0
@ -589,6 +678,47 @@ tox provision satisfied:
python3dist(inst)
result: 0
tox provision no minversion:
installed:
setuptools: 50
wheel: 1
tox: 3.5.3
tox-current-env: 0.0.6
toxenv:
- py3
setup.py: |
from setuptools import setup
setup(
name='test',
version='0.1',
)
tox.ini: |
[tox]
requires =
setuptools > 40
wheel > 2
expected:
- | # tox 3 with setuptools < 70
python3dist(setuptools) >= 40.8
python3dist(wheel)
python3dist(tox-current-env) >= 0.0.6
python3dist(setuptools) > 40.0
python3dist(wheel) > 2.0
- | # tox 4 with setuptools 70+
python3dist(setuptools) >= 40.8
python3dist(tox-current-env) >= 0.0.6
python3dist(setuptools) > 40.0
python3dist(wheel) > 2.0
python3dist(tox)
- | # tox 4 with setuptools < 70
python3dist(setuptools) >= 40.8
python3dist(wheel)
python3dist(tox-current-env) >= 0.0.6
python3dist(setuptools) > 40.0
python3dist(wheel) > 2.0
python3dist(tox)
result: 0
Default build system, unmet deps in requirements file:
installed:
setuptools: 50
@ -638,14 +768,21 @@ Default build system, met deps in requirements file:
SQLAlchemy>=1.0.10,<1.1.0 # Zebra protocol service
requirement_files:
- requirements.txt
expected: |
expected:
- | # setuptools 70+
((python3dist(lxml) < 3.7 or python3dist(lxml) > 3.7) with python3dist(lxml) >= 2.3)
python3dist(ncclient)
(python3dist(cryptography) < 1.5.2 or python3dist(cryptography) > 1.5.2)
python3dist(paramiko)
(python3dist(sqlalchemy) < 1.1~~ with python3dist(sqlalchemy) >= 1.0.10)
python3dist(setuptools) >= 40.8
- | # setuptools < 70
((python3dist(lxml) < 3.7 or python3dist(lxml) > 3.7) with python3dist(lxml) >= 2.3)
python3dist(ncclient)
(python3dist(cryptography) < 1.5.2 or python3dist(cryptography) > 1.5.2)
python3dist(paramiko)
(python3dist(sqlalchemy) < 1.1~~ with python3dist(sqlalchemy) >= 1.0.10)
python3dist(setuptools) >= 40.8
python3dist(wheel)
python3dist(wheel)
result: 0
@ -811,7 +948,12 @@ Pre-releases are accepted:
"cffi",
]
build-backend = "setuptools.build_meta"
expected: |
expected:
- | # setuptools 70+
python3dist(setuptools)
python3dist(wheel)
python3dist(cffi)
- | # setuptools < 70
python3dist(setuptools)
python3dist(wheel)
python3dist(cffi)
@ -830,9 +972,11 @@ Stdout from wrapped subprocess does not appear in output:
os.system('echo LEAK?')
from setuptools import setup
setup(name='test', version='0.1')
expected: |
expected:
- | # setuptools 70+
python3dist(setuptools) >= 40.8
- | # setuptools < 70
python3dist(setuptools) >= 40.8
python3dist(wheel)
python3dist(wheel)
result: 0
@ -853,7 +997,11 @@ pyproject.toml with runtime dependencies:
"foo",
'importlib-metadata; python_version<"3.8"',
]
expected: |
expected:
- | # setuptools 70+
python3dist(setuptools)
python3dist(foo)
- | # setuptools < 70
python3dist(setuptools)
python3dist(wheel)
python3dist(foo)
@ -867,7 +1015,7 @@ pyproject.toml with runtime dependencies and partially selected extras:
tomli: 1
extras:
- tests
pyproject.toml: |
pyproject.toml: &pyproject_with_extras |
[build-system]
requires = ["setuptools"]
build-backend = "setuptools.build_meta"
@ -881,7 +1029,13 @@ pyproject.toml with runtime dependencies and partially selected extras:
[project.optional-dependencies]
tests = ["pytest>=5", "pytest-mock"]
docs = ["sphinx", "python-docs-theme"]
expected: |
expected:
- | # setuptools 70+
python3dist(setuptools)
python3dist(foo)
python3dist(pytest) >= 5
python3dist(pytest-mock)
- | # setuptools < 70
python3dist(setuptools)
python3dist(wheel)
python3dist(foo)
@ -912,7 +1066,15 @@ Self-referencing extras (sooner):
tests = pytest>=5; pytest-mock
docs = sphinx; python-docs-theme
dev = my_package[docs,tests]
expected: |
expected:
- | # setuptools 70+
python3dist(setuptools)
python3dist(foo)
python3dist(sphinx)
python3dist(python-docs-theme)
python3dist(pytest) >= 5
python3dist(pytest-mock)
- | # setuptools < 70
python3dist(setuptools)
python3dist(wheel)
python3dist(foo)
@ -945,7 +1107,15 @@ Self-referencing extras (later):
tests = pytest>=5; pytest-mock
docs = sphinx; python-docs-theme
xdev = my_package[docs,tests]
expected: |
expected:
- | # setuptools 70+
python3dist(setuptools)
python3dist(foo)
python3dist(sphinx)
python3dist(python-docs-theme)
python3dist(pytest) >= 5
python3dist(pytest-mock)
- | # setuptools < 70
python3dist(setuptools)
python3dist(wheel)
python3dist(foo)
@ -977,7 +1147,15 @@ Self-referencing extras (maze):
forward = my_package[backward]; forwarddep
backward = my_package[left,right]; backwarddep
never = my_package[forward]; neverdep
expected: |
expected:
- | # setuptools 70+
python3dist(setuptools)
python3dist(backwarddep)
python3dist(forwarddep)
python3dist(leftdep)
python3dist(rightdep)
python3dist(startdep)
- | # setuptools < 70
python3dist(setuptools)
python3dist(wheel)
python3dist(backwarddep)
@ -1023,3 +1201,410 @@ config_settings:
expected: |
python3dist(test-config-setting)
result: 0
pyproject.toml with runtime dependencies read from it:
skipif: not SETUPTOOLS_60
read_pyproject_dependencies: true
installed:
setuptools: 50
wheel: 1
tomli: 1
pyproject.toml: |
[build-system]
requires = ["setuptools"]
build-backend = "setuptools.build_meta"
[project]
name = "my_package"
version = "0.1"
dependencies = [
"foo",
'importlib-metadata; python_version<"3.8"',
]
expected:
- | # setuptools 70+
python3dist(setuptools)
python3dist(foo)
- | # setuptools < 70
python3dist(setuptools)
python3dist(wheel)
python3dist(foo)
result: 0
pyproject.toml with extras - only runtime dependencies read from it:
skipif: not SETUPTOOLS_60
read_pyproject_dependencies: true
installed:
setuptools: 50
wheel: 1
tomli: 1
pyproject.toml: *pyproject_with_extras
expected:
- | # setuptools 70+
python3dist(setuptools)
python3dist(foo)
- | # setuptools < 70
python3dist(setuptools)
python3dist(wheel)
python3dist(foo)
result: 0
pyproject.toml with runtime dependencies and partially selected extras read from it:
skipif: not SETUPTOOLS_60
read_pyproject_dependencies: true
installed:
setuptools: 50
wheel: 1
tomli: 1
extras:
- tests
pyproject.toml: *pyproject_with_extras
expected:
- | # setuptools 70+
python3dist(setuptools)
python3dist(foo)
python3dist(pytest) >= 5
python3dist(pytest-mock)
- | # setuptools < 70
python3dist(setuptools)
python3dist(wheel)
python3dist(foo)
python3dist(pytest) >= 5
python3dist(pytest-mock)
result: 0
pyproject.toml with runtime dependencies and all extras read from it:
skipif: not SETUPTOOLS_60
read_pyproject_dependencies: true
installed:
setuptools: 50
wheel: 1
tomli: 1
extras:
- tests
- docs
pyproject.toml: *pyproject_with_extras
expected:
- | # setuptools 70+
python3dist(setuptools)
python3dist(foo)
python3dist(pytest) >= 5
python3dist(pytest-mock)
python3dist(sphinx)
python3dist(python-docs-theme)
- | # setuptools < 70
python3dist(setuptools)
python3dist(wheel)
python3dist(foo)
python3dist(pytest) >= 5
python3dist(pytest-mock)
python3dist(sphinx)
python3dist(python-docs-theme)
result: 0
pyproject.toml without dependencies:
skipif: not SETUPTOOLS_60
read_pyproject_dependencies: true
installed:
setuptools: 50
wheel: 1
tomli: 1
pyproject.toml: |
[build-system]
requires = ["setuptools"]
build-backend = "setuptools.build_meta"
[project]
name = "my_package"
version = "0.1"
expected:
- | # setuptools 70+
python3dist(setuptools)
- | # setuptools < 70
python3dist(setuptools)
python3dist(wheel)
result: 0
pyproject.toml without project table:
skipif: not SETUPTOOLS_60
read_pyproject_dependencies: true
installed:
setuptools: 50
wheel: 1
pyproject.toml: |
[build-system]
requires = ["setuptools"]
build-backend = "setuptools.build_meta"
except: ValueError
no pyproject.toml:
read_pyproject_dependencies: true
installed:
setuptools: 50
wheel: 1
except: FileNotFoundError
pyproject.toml with dynamic dependencies:
skipif: not SETUPTOOLS_60
read_pyproject_dependencies: true
installed:
setuptools: 50
wheel: 1
tomli: 1
pyproject.toml: |
[build-system]
requires = ["setuptools"]
build-backend = "setuptools.build_meta"
[project]
name = "my_package"
version = "0.1"
dynamic = ["dependencies"]
[tool.setuptools.dynamic]
dependencies = { file = ["deps.txt"] }
deps.txt: |
foo < 7.0
sphinx
except: ValueError
pyproject.toml with dynamic optional dependencies:
skipif: not SETUPTOOLS_60
read_pyproject_dependencies: true
installed:
setuptools: 50
wheel: 1
tomli: 1
extras:
- docs
pyproject.toml: |
[build-system]
requires = ["setuptools"]
build-backend = "setuptools.build_meta"
[project]
name = "my_package"
version = "0.1"
dynamic = ["optional-dependencies"]
[tool.setuptools.dynamic.optional-dependencies.docs]
file = ["deps.txt"]
deps.txt: |
sphinx~=7.0.1
except: ValueError
pyproject.toml with dynamic table and no deps:
skipif: not SETUPTOOLS_60
read_pyproject_dependencies: true
installed:
setuptools: 50
wheel: 1
tomli: 1
pyproject.toml: |
[build-system]
requires = ["setuptools"]
build-backend = "setuptools.build_meta"
[project]
name = "my_package"
version = "0.1"
dynamic = ["readme"]
[tool.setuptools.dynamic]
readme = { file = ["readme.txt"] }
readme.txt: |
nothing interesting here
expected:
- | # setuptools 70+
python3dist(setuptools)
- | # setuptools < 70
python3dist(setuptools)
python3dist(wheel)
result: 0
pyproject.toml with self-referencing extras:
skipif: not SETUPTOOLS_60
read_pyproject_dependencies: true
installed:
setuptools: 50
wheel: 1
tomli: 1
extras:
- test
pyproject.toml: |
[build-system]
requires = ["setuptools"]
build-backend = "setuptools.build_meta"
[project]
name = "contourpy"
version = "0.1"
dependencies = ["numpy >= 1.23"]
[project.optional-dependencies]
bokeh = ["bokeh", "selenium"]
test = ["contourpy[test-no-images]", "matplotlib", "Pillow"]
test-no-images = ["pytest", "pytest-rerunfailures", "wurlitzer"]
expected:
- | # setuptools 70+
python3dist(setuptools)
python3dist(numpy) >= 1.23
python3dist(matplotlib)
python3dist(pillow)
python3dist(pytest)
python3dist(pytest-rerunfailures)
python3dist(wurlitzer)
- | # setuptools < 70
python3dist(setuptools)
python3dist(wheel)
python3dist(numpy) >= 1.23
python3dist(matplotlib)
python3dist(pillow)
python3dist(pytest)
python3dist(pytest-rerunfailures)
python3dist(wurlitzer)
result: 0
pyproject.toml with dependency-groups not requested:
use_build_system: false
pyproject.toml: |
[dependency-groups]
tests = ["pytest>=5", "pytest-mock"]
docs = ["sphinx", "python-docs-theme"]
expected: "\n"
result: 0
pyproject.toml with dependency-groups and build system:
skipif: not SETUPTOOLS_60
use_build_system: true
installed:
setuptools: 50
wheel: 1
tomli: 1
dependency_groups:
- tests
pyproject.toml: |
[build-system]
requires = ["setuptools"]
build-backend = "setuptools.build_meta"
[project]
name = "my_package"
version = "0.1"
[dependency-groups]
tests = ["pytest>=5", "pytest-mock"]
docs = ["sphinx", "python-docs-theme"]
expected:
- | # setuptools 70+
python3dist(setuptools)
python3dist(pytest) >= 5
python3dist(pytest-mock)
- | # setuptools < 70
python3dist(setuptools)
python3dist(wheel)
python3dist(pytest) >= 5
python3dist(pytest-mock)
result: 0
pyproject.toml with dependency-groups one requested:
use_build_system: false
dependency_groups:
- tests
pyproject.toml: |
[dependency-groups]
tests = ["pytest>=5", "pytest-mock"]
docs = ["sphinx", "python-docs-theme"]
expected: |
python3dist(pytest) >= 5
python3dist(pytest-mock)
result: 0
pyproject.toml with dependency-groups two requested:
use_build_system: false
dependency_groups:
- tests
- docs
pyproject.toml: |
[dependency-groups]
tests = ["pytest>=5", "pytest-mock"]
docs = ["sphinx", "python-docs-theme"]
expected: |
python3dist(pytest) >= 5
python3dist(pytest-mock)
python3dist(sphinx)
python3dist(python-docs-theme)
result: 0
pyproject.toml with dependency-groups two requested via comma:
use_build_system: false
dependency_groups:
- tests,docs
pyproject.toml: |
[dependency-groups]
tests = ["pytest>=5", "pytest-mock"]
docs = ["sphinx", "python-docs-theme"]
expected: |
python3dist(pytest) >= 5
python3dist(pytest-mock)
python3dist(sphinx)
python3dist(python-docs-theme)
result: 0
pyproject.toml with include-group:
use_build_system: false
dependency_groups:
- tests_docs
pyproject.toml: |
[dependency-groups]
tests = ["pytest>=5", "pytest-mock"]
docs = ["sphinx", "python-docs-theme"]
typing = ["mypy"]
tests-docs = [{include-group = "tests"}, {include-group = "docs"}, "pytest-sphinx"]
expected: |
python3dist(pytest) >= 5
python3dist(pytest-mock)
python3dist(sphinx)
python3dist(python-docs-theme)
python3dist(pytest-sphinx)
result: 0
pyproject.toml with dependency-groups nonexisting requested:
use_build_system: false
dependency_groups:
- typing
pyproject.toml: |
[dependency-groups]
tests = ["pytest>=5", "pytest-mock"]
docs = ["sphinx", "python-docs-theme"]
except: LookupError
tox with dependency_groups:
skipif: not (SETUPTOOLS_60 and TOX_4_22)
installed:
setuptools: 50
wheel: 1
tox: 4.22
tox-current-env: 0.0.14
toxenv:
- py3
pyproject.toml: |
[build-system]
requires = ["setuptools"]
build-backend = "setuptools.build_meta"
[project]
name = "my_package"
version = "0.1"
[dependency-groups]
tests = ["pytest>=5", "pytest-mock"]
docs = ["sphinx", "python-docs-theme"]
[tool.tox]
requires = ["tox>=4.22"]
[tool.tox.env_run_base]
dependency_groups = ["tests"]
commands = [["pytest"]]
expected:
- | # setuptools 70+
python3dist(setuptools)
python3dist(tox-current-env) >= 0.0.6
python3dist(tox) >= 4.22
python3dist(tox)
python3dist(pytest) >= 5
python3dist(pytest-mock)
- | # setuptools < 70
python3dist(setuptools)
python3dist(wheel)
python3dist(tox-current-env) >= 0.0.6
python3dist(tox) >= 4.22
python3dist(tox)
python3dist(pytest) >= 5
python3dist(pytest-mock)
result: 0

@ -2,6 +2,7 @@ import argparse
import fnmatch
import json
import os
import re
from collections import defaultdict
from keyword import iskeyword
@ -11,9 +12,15 @@ from importlib.metadata import Distribution
# From RPM's build/files.c strtokWithQuotes delim argument
RPM_FILES_DELIMETERS = ' \n\t'
RPM_GLOB_SYMBOLS = '[]{}*?!'
# Combined for escape_rpm_path_4_19()
RPM_SPECIAL_SYMBOLS = RPM_FILES_DELIMETERS + RPM_GLOB_SYMBOLS + '"' + "\\"
RPM_ESCAPE_REGEX = re.compile(f"([{re.escape(RPM_SPECIAL_SYMBOLS)}])")
# See the comment in the macro that wraps this script
RPM_PERCENTAGES_COUNT = int(os.getenv('RPM_PERCENTAGES_COUNT', '2'))
RPM_FILES_ESCAPE = os.getenv('RPM_FILES_ESCAPE', '4.19')
PYCACHED_SUFFIX = '{,.opt-?}.pyc'
# RPM hardcodes the lists of manpage extensions and directories,
# so we have to maintain separate ones :(
@ -118,8 +125,9 @@ def pycached(script, python_version):
"""
assert script.suffix == ".py"
pyver = "".join(python_version.split(".")[:2])
pycname = f"{script.stem}.cpython-{pyver}{{,.opt-?}}.pyc"
pycname = f"{script.stem}.cpython-{pyver}{PYCACHED_SUFFIX}"
pyc = pycache_dir(script) / pycname
pyc.glob_suffix_len = len(PYCACHED_SUFFIX)
return [script, pyc]
@ -212,10 +220,12 @@ def normalize_manpage_filename(prefix, path):
if fnmatch.fnmatch(str(path.parent), mandir) and path.name != "dir":
# "abc.1.gz2" -> "abc.1*"
if path.suffix[1:] in MANPAGE_EXTENSIONS:
return BuildrootPath(path.parent / (path.stem + "*"))
path = BuildrootPath(path.parent / (path.stem + "*"))
# "abc.1 -> abc.1*"
else:
return BuildrootPath(path.parent / (path.name + "*"))
path = BuildrootPath(path.parent / (path.name + "*"))
path.glob_suffix_len = 1
return path
else:
return path
@ -345,7 +355,7 @@ def classify_paths(
}
license_files = metadata.get_all('License-File')
license_directory = distinfo / 'licenses' # See PEP 369 "Root License Directory"
license_directory = distinfo / 'licenses' # See PEP 639 "Root License Directory"
# setuptools was the first known build backend to implement License-File.
# Unfortunately they don't put licenses to the license directory (yet):
# https://github.com/pypa/setuptools/issues/3596
@ -424,60 +434,139 @@ def classify_paths(
return paths
def escape_rpm_path(path):
def escape_rpm_path_4_19(path):
r"""
Escape special characters in string-paths or BuildrootPaths, RPM >= 4.19
E.g. a space in path otherwise makes RPM think it's multiple paths,
unless we escape it.
Or a literal % symbol in path might be expanded as a macro if not escaped by %%.
See https://github.com/rpm-software-management/rpm/pull/2103
and https://github.com/rpm-software-management/rpm/pull/2206
If the path ends with a glob produced by our other functions,
we cannot escape that part.
The BuildrootPath.glob_suffix_len attribute is used to indicate such globs.
When such suffix exists, it is not escaped.
Examples:
>>> escape_rpm_path_4_19(BuildrootPath('/usr/lib/python3.9/site-packages/setuptools'))
'/usr/lib/python3.9/site-packages/setuptools'
>>> escape_rpm_path_4_19('/usr/lib/python3.9/site-packages/setuptools/script (dev).tmpl')
'/usr/lib/python3.9/site-packages/setuptools/script\\ (dev).tmpl'
>>> escape_rpm_path_4_19('/usr/share/data/100%valid.path')
'/usr/share/data/100%%valid.path'
>>> escape_rpm_path_4_19('/usr/share/data/100 % valid.path')
'/usr/share/data/100\\ %%\\ valid.path'
>>> escape_rpm_path_4_19('/usr/share/data/1000 %% valid.path')
'/usr/share/data/1000\\ %%%%\\ valid.path'
>>> escape_rpm_path_4_19('/usr/share/data/spaces and "quotes" and ?')
'/usr/share/data/spaces\\ and\\ \\"quotes\\"\\ and\\ \\?'
>>> escape_rpm_path_4_19('/usr/share/data/spaces and [square brackets]')
'/usr/share/data/spaces\\ and\\ \\[square\\ brackets\\]'
>>> path = BuildrootPath('/whatever/__pycache__/bar.cpython-38{,.opt-?}.pyc')
>>> path.glob_suffix_len = len('{,.opt-?}.pyc')
>>> escape_rpm_path_4_19(path)
'/whatever/__pycache__/bar.cpython-38{,.opt-?}.pyc'
>>> path = BuildrootPath('/spa ces/__pycache__/bar.cpython-38{,.opt-?}.pyc')
>>> path.glob_suffix_len = len('{,.opt-?}.pyc')
>>> escape_rpm_path_4_19(path)
'/spa\\ ces/__pycache__/bar.cpython-38{,.opt-?}.pyc'
>>> path = BuildrootPath('/usr/man/man5/ipykernel.5*')
>>> path.glob_suffix_len = 1
>>> escape_rpm_path_4_19(path)
'/usr/man/man5/ipykernel.5*'
"""
Escape special characters in string-paths or BuildrootPaths
glob_suffix_len = getattr(path, "glob_suffix_len", 0)
suffix = ""
path = str(path)
if glob_suffix_len:
suffix = path[-glob_suffix_len:]
path = path[:-glob_suffix_len]
if "%" in path:
path = path.replace("%", "%%")
# Prepend all matched/special characters (\1) with a backslash (escaped, hence \\):
return RPM_ESCAPE_REGEX.sub(r'\\\1', path) + suffix
def escape_rpm_path_4_18(path):
"""
Escape special characters in string-paths or BuildrootPaths, RPM < 4.19
E.g. a space in path otherwise makes RPM think it's multiple paths,
unless we put it in "quotes".
Or a literal % symbol in path might be expanded as a macro if not escaped.
Due to limitations in RPM,
Due to limitations in RPM < 4.19,
some paths with spaces and other special characters are not supported.
See this thread http://lists.rpm.org/pipermail/rpm-list/2021-June/002048.html
Examples:
>>> escape_rpm_path(BuildrootPath('/usr/lib/python3.9/site-packages/setuptools'))
>>> escape_rpm_path_4_18(BuildrootPath('/usr/lib/python3.9/site-packages/setuptools'))
'/usr/lib/python3.9/site-packages/setuptools'
>>> escape_rpm_path('/usr/lib/python3.9/site-packages/setuptools/script (dev).tmpl')
>>> escape_rpm_path_4_18('/usr/lib/python3.9/site-packages/setuptools/script (dev).tmpl')
'"/usr/lib/python3.9/site-packages/setuptools/script (dev).tmpl"'
>>> escape_rpm_path('/usr/share/data/100%valid.path')
'/usr/share/data/100%%valid.path'
>>> escape_rpm_path_4_18('/usr/share/data/100%valid.path')
'/usr/share/data/100%%%%%%%%valid.path'
>>> escape_rpm_path('/usr/share/data/100 % valid.path')
'"/usr/share/data/100 %% valid.path"'
>>> escape_rpm_path_4_18('/usr/share/data/100 % valid.path')
'"/usr/share/data/100 %%%%%%%% valid.path"'
>>> escape_rpm_path('/usr/share/data/1000 %% valid.path')
'"/usr/share/data/1000 %%%% valid.path"'
>>> escape_rpm_path_4_18('/usr/share/data/1000 %% valid.path')
'"/usr/share/data/1000 %%%%%%%%%%%%%%%% valid.path"'
>>> escape_rpm_path('/usr/share/data/spaces and "quotes"')
>>> escape_rpm_path_4_18('/usr/share/data/spaces and "quotes"')
Traceback (most recent call last):
...
NotImplementedError: ...
>>> escape_rpm_path('/usr/share/data/spaces and [square brackets]')
>>> escape_rpm_path_4_18('/usr/share/data/spaces and [square brackets]')
Traceback (most recent call last):
...
NotImplementedError: ...
"""
orig_path = path = str(path)
if "%" in path:
path = path.replace("%", "%" * RPM_PERCENTAGES_COUNT)
# Escaping an actual percentage sign in path by 8 signs
# has been verified in RPM 4.16 and 4.17:
path = path.replace("%", "%" * 8)
if any(symbol in path for symbol in RPM_FILES_DELIMETERS):
if '"' in path:
# As far as we know, RPM cannot list such file individually
# As far as we know, RPM < 4.19 cannot list such file individually
# See this thread http://lists.rpm.org/pipermail/rpm-list/2021-June/002048.html
raise NotImplementedError(f'" symbol in path with spaces is not supported by %pyproject_save_files: {orig_path!r}')
raise NotImplementedError(f'" symbol in path with spaces is not supported by %pyproject_save_files on RPM < 4.19: {orig_path!r}')
if "[" in path or "]" in path:
# See https://bugzilla.redhat.com/show_bug.cgi?id=1990879
# and https://github.com/rpm-software-management/rpm/issues/1749
raise NotImplementedError(f'[ or ] symbol in path with spaces is not supported by %pyproject_save_files: {orig_path!r}')
raise NotImplementedError(f'[ or ] symbol in path with spaces is not supported by %pyproject_save_files on RPM < 4.19: {orig_path!r}')
return f'"{path}"'
return path
if RPM_FILES_ESCAPE == "4.19":
escape_rpm_path = escape_rpm_path_4_19
elif RPM_FILES_ESCAPE == "4.18":
escape_rpm_path = escape_rpm_path_4_18
else:
raise RuntimeError("RPM_FILES_ESCAPE must be 4.18 or 4.19")
def generate_file_list(paths_dict, module_globs, include_others=False):
"""
This function takes the classified paths_dict and turns it into lines
@ -673,7 +762,7 @@ def load_parsed_record(pyproject_record):
content = json.load(pyproject_record_file)
if len(content) > 1:
raise FileExistsError("%pyproject install has found more than one *.dist-info/RECORD file. "
raise FileExistsError("%pyproject_install has found more than one *.dist-info/RECORD file. "
"Currently, %pyproject_save_files supports only one wheel → one file list mapping. "
"Feel free to open a bugzilla for pyproject-rpm-macros and describe your usecase.")
@ -693,12 +782,15 @@ def dist_metadata(buildroot, record_path):
return dist.metadata
def pyproject_save_files_and_modules(buildroot, sitelib, sitearch, python_version, pyproject_record, prefix, varargs):
def pyproject_save_files_and_modules(buildroot, sitelib, sitearch, python_version, pyproject_record, prefix, assert_license, varargs):
"""
Takes arguments from the %{pyproject_save_files} macro
Returns tuple: list of paths for the %files section and list of module names
for the %check section
Raises ValueError when assert_license is true and no License-File (PEP 639)
is found.
"""
# On 32 bit architectures, sitelib equals to sitearch
# This saves us browsing one directory twice
@ -710,11 +802,15 @@ def pyproject_save_files_and_modules(buildroot, sitelib, sitearch, python_versio
final_file_list = []
final_module_list = []
# we assume OK when not asserting
license_ok = not assert_license
for record_path, files in parsed_records.items():
metadata = dist_metadata(buildroot, record_path)
paths_dict = classify_paths(
record_path, files, metadata, sitedirs, python_version, prefix
)
license_ok = license_ok or bool(paths_dict["metadata"]["licenses"])
final_file_list.extend(
generate_file_list(paths_dict, globs, include_auto)
@ -723,6 +819,15 @@ def pyproject_save_files_and_modules(buildroot, sitelib, sitearch, python_versio
generate_module_list(paths_dict, globs)
)
if not license_ok:
raise ValueError(
"No License-File (PEP 639) in upstream metadata found. "
"Adjust the upstream metadata "
"if the project's build backend supports PEP 639 "
"or use `%pyproject_save_files -L` "
"and include the %license file in %files manually."
)
return final_file_list, final_module_list
@ -734,6 +839,7 @@ def main(cli_args):
cli_args.python_version,
cli_args.pyproject_record,
cli_args.prefix,
cli_args.assert_license,
cli_args.varargs,
)
@ -747,7 +853,7 @@ def argparser():
prog="%pyproject_save_files",
add_help=False,
# custom usage to add +auto
usage="%(prog)s MODULE_GLOB [MODULE_GLOB ...] [+auto]",
usage="%(prog)s [-l|-L] MODULE_GLOB [MODULE_GLOB ...] [+auto]",
)
parser.add_argument(
'--help', action='help',
@ -763,6 +869,14 @@ def argparser():
r.add_argument("--python-version", type=str, required=True, help=argparse.SUPPRESS)
r.add_argument("--pyproject-record", type=PosixPath, required=True, help=argparse.SUPPRESS)
r.add_argument("--prefix", type=PosixPath, required=True, help=argparse.SUPPRESS)
parser.add_argument(
"-l", "--assert-license", action="store_true", default=False,
help="Fail when no License-File (PEP 639) is found.",
)
parser.add_argument(
"-L", "--no-assert-license", action="store_false", dest="assert_license",
help="Don't fail when no License-File (PEP 639) is found (the default).",
)
parser.add_argument(
"varargs", nargs="+", metavar="MODULE_GLOB",
help="Shell-like glob matching top-level module names to save into %%{pyproject_files}",

@ -457,7 +457,7 @@ classified:
- /usr/lib/python3.7/site-packages/comic2pdf-3.1.0.dist-info/top_level.txt
- /usr/lib/python3.7/site-packages/comic2pdf-3.1.0.dist-info/zip-safe
licenses: []
modules: []
modules: {}
other:
files:
- /usr/bin/comic2pdf.py

@ -6,16 +6,36 @@ import pytest
import setuptools
import yaml
from pyproject_buildrequires import generate_requires
from pyproject_buildrequires import generate_requires, load_pyproject
SETUPTOOLS_VERSION = packaging.version.parse(setuptools.__version__)
SETUPTOOLS_60 = SETUPTOOLS_VERSION >= packaging.version.parse('60')
try:
import tox
except ImportError:
TOX_4_22 = False
else:
TOX_VERSION = packaging.version.parse(tox.__version__)
TOX_4_22 = TOX_VERSION >= packaging.version.parse('4.22')
testcases = {}
with Path(__file__).parent.joinpath('pyproject_buildrequires_testcases.yaml').open() as f:
testcases = yaml.safe_load(f)
@pytest.fixture(autouse=True)
def clear_pyproject_data():
"""
Clear pyproject data before each test.
In reality we build one RPM package at a time, so we can keep the once-loaded
pyproject.toml contents.
When testing, the cached data would leak the once-loaded data to all the
following test cases.
"""
load_pyproject.cache_clear()
@pytest.mark.parametrize('case_name', testcases)
def test_data(case_name, capfd, tmp_path, monkeypatch):
case = testcases[case_name]
@ -51,6 +71,7 @@ def test_data(case_name, capfd, tmp_path, monkeypatch):
requirement_files = case.get('requirement_files', [])
requirement_files = [open(f) for f in requirement_files]
use_build_system = case.get('use_build_system', True)
read_pyproject_dependencies = case.get('read_pyproject_dependencies', False)
try:
generate_requires(
get_installed_version=get_installed_version,
@ -58,10 +79,12 @@ def test_data(case_name, capfd, tmp_path, monkeypatch):
build_wheel=case.get('build_wheel', False),
wheeldir=str(wheeldir),
extras=case.get('extras', []),
dependency_groups=case.get('dependency_groups', []),
toxenv=case.get('toxenv', None),
generate_extras=case.get('generate_extras', False),
requirement_files=requirement_files,
use_build_system=use_build_system,
read_pyproject_dependencies=read_pyproject_dependencies,
output=output,
config_settings=case.get('config_settings'),
)

@ -25,6 +25,21 @@ TEST_RECORDS = yaml_data["records"]
TEST_METADATAS = yaml_data["metadata"]
# insert glob_suffix_len for .pyc files and man pages globs
for paths_dict in EXPECTED_DICT.values():
for modules in paths_dict["modules"].values():
for module in modules:
for idx, file in enumerate(module["files"]):
if file.endswith(".pyc"):
module["files"][idx] = BuildrootPath(file)
module["files"][idx].glob_suffix_len = len("{,.opt-?}.pyc")
if "other" in paths_dict and "files" in paths_dict["other"]:
for idx, file in enumerate(paths_dict["other"]["files"]):
if file.endswith("*"):
paths_dict["other"]["files"][idx] = BuildrootPath(file)
paths_dict["other"]["files"][idx].glob_suffix_len = len("*")
@pytest.fixture
def tldr_root(tmp_path):
prepare_pyproject_record(tmp_path, package="tldr")

@ -1,5 +1,6 @@
Name: pyproject-rpm-macros
Summary: RPM macros for PEP 517 Python packages
# SPDX
License: MIT
%bcond tests 1
@ -13,7 +14,7 @@ License: MIT
# Increment Y and reset Z when new macros or features are added
# Increment Z when this is a bugfix or a cosmetic change
# Dropping support for EOL Fedoras is *not* considered a breaking change
Version: 1.9.0
Version: 1.16.2
Release: 1%{?dist}
# Macro files
@ -62,14 +63,6 @@ BuildRequires: python3dist(tox-current-env) >= 0.0.6
%endif
BuildRequires: python3dist(wheel)
BuildRequires: (python3dist(tomli) if python3 < 3.11)
# RHEL 9: We also run pytest with Python 3.11
BuildRequires: python3.11dist(pytest)
BuildRequires: python3.11dist(pyyaml)
BuildRequires: python3.11dist(packaging)
BuildRequires: python3.11dist(pip)
BuildRequires: python3.11dist(setuptools)
BuildRequires: python3.11dist(wheel)
%endif
# We build on top of those:
@ -85,6 +78,13 @@ Requires: (pyproject-srpm-macros = %{?epoch:%{epoch}:}%{version}-%{release
Requires: /usr/bin/find
Requires: /usr/bin/sed
# This package requires the %%generate_buildrequires functionality.
# It has been introduced in RPM 4.15 (4.14.90 is the alpha of 4.15).
# What we need is rpmlib(DynamicBuildRequires), but that is impossible to (Build)Require.
# Also, we need to avoid 4.19.90..4.19.91-7 due to rhbz#2284187
Requires: ((rpm-build >= 4.14.90 with (rpm-build < 4.19.90 or rpm-build >= 4.19.91-8)) if rpm-build)
BuildRequires: rpm-build >= 4.14.90
%description
These macros allow projects that follow the Python packaging specifications
to be packaged as RPMs.
@ -103,6 +103,7 @@ which only work with setup.py.
%package -n pyproject-srpm-macros
Summary: Minimal implementation of %%pyproject_buildrequires
Requires: (pyproject-rpm-macros = %{?epoch:%{epoch}:}%{version}-%{release} if pyproject-rpm-macros)
Requires: (rpm-build >= 4.14.90 if rpm-build)
%description -n pyproject-srpm-macros
This package contains a minimal implementation of %%pyproject_buildrequires.
@ -117,6 +118,9 @@ takes precedence.
%setup -c -T
cp -p %{sources} .
%generate_buildrequires
# nothing to do, this is here just to assert we have that functionality
%build
# nothing to do, sources are not buildable
@ -145,10 +149,6 @@ test "$signature1" != ""
export HOSTNAME="rpmbuild" # to speedup tox in network-less mock, see rhbz#1856356
%pytest -vv --doctest-modules %{?with_pytest_xdist:-n auto} %{!?with_tox_tests:-k "not tox"}
# RHEL 9 only:
%global __pytest %{__pytest}-3.11
%pytest -vv --doctest-modules -k "not tox"
# brp-compress is provided as an argument to get the right directory macro expansion
%{python3} compare_mandata.py -f %{_rpmconfigdir}/brp-compress
%endif
@ -173,8 +173,80 @@ export HOSTNAME="rpmbuild" # to speedup tox in network-less mock, see rhbz#1856
%changelog
* Wed Nov 13 2024 Miro Hrončok <mhroncok@redhat.com> - 1.16.2-1
- Fix one remaining test for setuptools 70+
* Thu Nov 07 2024 Miro Hrončok <miro@hroncok.cz> - 1.16.1-1
- Support for setuptools 70+
- wheel is no longer generated as a dependency of the default build system
* Mon Nov 04 2024 Miro Hrončok <mhroncok@redhat.com> - 1.16.0-1
- %%pyproject_buildrequires: Add support for dependency groups (PEP 735), via the -g flag
- This is implied when used tox testenvs depend on dependency groups (requires tox 4.22+)
- Fixes: rhbz#2318849
* Thu Oct 03 2024 Karolina Surma <ksurma@redhat.com> - 1.15.1-1
- Fix handling of self-referencing extras when reading pyproject.toml
* Tue Sep 17 2024 Python Maint <python-maint@redhat.com> - 1.15.0-1
- Add a possibility to read runtime requirements from pyproject.toml [project] table
- Fixes: rhbz#2261939
- Don't generate a dependency on pip when %%pyproject_buildrequires -N is used
- Fixes: rhbz#2294510
- Even when %%_auto_set_build_flags is disabled, set all compiler flags when building wheels
- Fixes: rhbz#2293616
* Tue Jul 23 2024 Miro Hrončok <mhroncok@redhat.com> - 1.14.0-1
- Add a provisional RPM Declarative Buildsystem (RPM 4.20+)
* Fri Jul 19 2024 Fedora Release Engineering <releng@fedoraproject.org> - 1.13.0-2
- Rebuilt for https://fedoraproject.org/wiki/Fedora_41_Mass_Rebuild
* Tue Jul 02 2024 Miro Hrončok <mhroncok@redhat.com> - 1.13.0-1
- Properly escape weird characters from paths in %%{pyproject_files} (RPM 4.19+ only)
- Revert the temporary workaround for RPM 4.20 alpha 2 leaking \x1f (unit separators)
- Fixes: rhbz#1990879
* Tue Jun 25 2024 Cristian Le <fedora@lecris.me> - 1.12.2-1
- %%pyproject_extras_subpkg: Allow passing -a or -A to %%python_extras_subpkg
* Tue Jun 04 2024 Miro Hrončok <mhroncok@redhat.com> - 1.12.1-1
- Add a temporary workaround for RPM 4.20 alpha 2 leaking \x1f (unit separators)
- Related: rhbz#2284187
* Fri Jan 26 2024 Miro Hrončok <miro@hroncok.cz> - 1.12.0-1
- Namespace pyproject-rpm-macros generated text files with %%{python3_pkgversion}
- That way, a single-spec can be used to build packages for multiple Python versions
- Fixes: rhbz#2209055
* Fri Jan 26 2024 Fedora Release Engineering <releng@fedoraproject.org> - 1.11.0-3
- Rebuilt for https://fedoraproject.org/wiki/Fedora_40_Mass_Rebuild
* Sun Jan 21 2024 Fedora Release Engineering <releng@fedoraproject.org> - 1.11.0-2
- Rebuilt for https://fedoraproject.org/wiki/Fedora_40_Mass_Rebuild
* Wed Sep 27 2023 Miro Hrončok <mhroncok@redhat.com> - 1.11.0-1
- Add the -l/-L flag to %%pyproject_save_files
- The -l flag can be used to assert at least 1 License-File was detected
- The -L flag explicitly disables this check (which remains the default)
- Prevent incorrect usage of %%pyproject_buildrequires -R with -x/-e/-t
- Fixes: rhbz#2244282
- Show a better error message when %%pyproject_install finds no wheel
- Fixes: rhbz#2242452
- Fix %%pyproject_buildrequires -w when the build backend is already installed and pip isn't
- Fixes: rhbz#2169855
* Wed Sep 13 2023 Python Maint <python-maint@redhat.com> - 1.10.0-1
- Add %%_pyproject_check_import_allow_no_modules for automated environments
- Fix handling of tox 4 provision without an explicit tox minversion
- Fixes: rhbz#2240590
* Fri Jul 21 2023 Fedora Release Engineering <releng@fedoraproject.org> - 1.9.0-2
- Rebuilt for https://fedoraproject.org/wiki/Fedora_39_Mass_Rebuild
* Wed May 31 2023 Maxwell G <maxwell@gtmx.me> - 1.9.0-1
- Allow passing config_settings to the build backend.
- Resolves: rhbz#2192581
* Wed May 31 2023 Miro Hrončok <mhroncok@redhat.com> - 1.8.1-1
- On Python older than 3.11, use tomli instead of deprecated toml
@ -185,12 +257,14 @@ export HOSTNAME="rpmbuild" # to speedup tox in network-less mock, see rhbz#1856
* Thu Apr 27 2023 Miro Hrončok <mhroncok@redhat.com> - 1.8.0-1
- %%pyproject_buildrequires: Add support for self-referential extras requirements
Fixes: rhbz#2171343
- Deprecate the provisional %%{pyproject_build_lib} macro
See https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org/thread/HMLOPAU3RZLXD4BOJHTIPKI3I4U6U7OE/
* Fri Mar 31 2023 Miro Hrončok <mhroncok@redhat.com> - 1.7.0-1
- %%pyproject_buildrequires: Redirect stdout to stderr via Shell
- Dependencies are recorded to a text file that is catted at the end
- Fixes: rhbz#2183519
* Mon Feb 13 2023 Lumír Balhar <lbalhar@redhat.com> - 1.6.3-1
- Remove .dist-info directory at the end of %%pyproject_buildrequires
@ -198,23 +272,31 @@ export HOSTNAME="rpmbuild" # to speedup tox in network-less mock, see rhbz#1856
* Wed Feb 08 2023 Lumír Balhar <lbalhar@redhat.com> - 1.6.2-1
- Improve detection of lang files
- Fixes: rhbz#2166295
* Fri Feb 03 2023 Miro Hrončok <mhroncok@redhat.com> - 1.6.1-1
- %%pyproject_buildrequires: Avoid leaking stdout from subprocesses
- Fixes: rhbz#2166888
* Fri Jan 20 2023 Miro Hrončok <miro@hroncok.cz> - 1.6.0-1
- Add pyproject-srpm-macros with a minimal %%pyproject_buildrequires macro
* Fri Jan 20 2023 Fedora Release Engineering <releng@fedoraproject.org> - 1.5.1-2
- Rebuilt for https://fedoraproject.org/wiki/Fedora_38_Mass_Rebuild
* Fri Jan 13 2023 Miro Hrončok <mhroncok@redhat.com> - 1.5.1-1
- Adjusts %%pyproject_buildrequires tests for tox 4
- Fixes: rhbz#2160687
* Mon Nov 28 2022 Miro Hrončok <mhroncok@redhat.com> - 1.5.0-1
- Use %%py3_test_envvars in %%tox when available
* Mon Sep 19 2022 Python Maint <python-maint@redhat.com> - 1.4.0-1
- %%pyproject_save_files: Support License-Files installed into the *Root License Directory* from PEP 369
- %%pyproject_save_files: Support License-Files installed into the *Root License Directory* from PEP 639
- Fixes: rhbz#2127946
- %%pyproject_check_import: Import only the modules whose top-level names
match any of the globs provided to %%pyproject_save_files
- Fixes: rhbz#2127958
* Tue Aug 30 2022 Otto Liljalaakso <otto.liljalaakso@iki.fi> - 1.3.4-1
- Fix typo in internal function name
@ -222,10 +304,15 @@ export HOSTNAME="rpmbuild" # to speedup tox in network-less mock, see rhbz#1856
* Tue Aug 09 2022 Karolina Surma <ksurma@redhat.com> - 1.3.3-1
- Don't fail %%pyproject_save_files '*' if no modules are detected
* Fri Jul 22 2022 Fedora Release Engineering <releng@fedoraproject.org> - 1.3.2-2
- Rebuilt for https://fedoraproject.org/wiki/Fedora_37_Mass_Rebuild
* Wed Jun 15 2022 Benjamin A. Beasley <code@musicinmybrain.net> - 1.3.2-1
- Update %%pyproject_build_lib to support setuptools 62.1.0 and later
- Fixes: rhbz#2097158
- %%pyproject_buildrequires: When extension modules are built,
support https://fedoraproject.org/wiki/Changes/Package_information_on_ELF_objects
- Fixes: rhbz#2097535
* Fri May 27 2022 Owen Taylor <otaylor@redhat.com> - 1.3.1-1
- %%pyproject_install: pass %%{_prefix} explicitly to pip install
@ -238,9 +325,11 @@ export HOSTNAME="rpmbuild" # to speedup tox in network-less mock, see rhbz#1856
prepare_metadata_for_build_wheel hook
When used, the wheel is built in %%pyproject_buildrequires
and information about runtime requires and extras is read from that wheel.
- Fixes: rhbz#2076994
* Tue Apr 12 2022 Miro Hrončok <mhroncok@redhat.com> - 1.1.0-1
- %%pyproject_save_files: Support nested directories in dist-info
- Fixes: rhbz#1985340
* Tue Mar 22 2022 Miro Hrončok <mhroncok@redhat.com> - 1.0.1-1
- Prefix paths of intermediate files (such as %%{pyproject_files}) with NVRA
@ -260,6 +349,7 @@ export HOSTNAME="rpmbuild" # to speedup tox in network-less mock, see rhbz#1856
* Wed Jan 19 2022 Karolina Surma <ksurma@redhat.com> - 0-54
- Include compressed manpages to the package if flag '+auto' is provided to %%pyproject_save_files
- Fixes: rhbz#2033254
* Fri Jan 14 2022 Miro Hrončok <mhroncok@redhat.com> - 0-53
- %%pyproject_buildrequires: Make -r (include runtime) the default, use -R to opt-out
@ -282,6 +372,7 @@ export HOSTNAME="rpmbuild" # to speedup tox in network-less mock, see rhbz#1856
* Sat Oct 16 2021 Miro Hrončok <mhroncok@redhat.com> - 0-48
- %%pyproject_buildrequires: Accept installed pre-releases for all requirements
- Fixes: rhbz#2014639
* Thu Sep 09 2021 Miro Hrončok <mhroncok@redhat.com> - 0-47
- %%pyproject_save_files: Expand the namespace error message, also display it with /
@ -289,6 +380,7 @@ export HOSTNAME="rpmbuild" # to speedup tox in network-less mock, see rhbz#1856
* Fri Jul 23 2021 Miro Hrončok <miro@hroncok.cz> - 0-46
- %%pyproject_buildrequires now fails when it encounters an invalid requirement
- Fixes: rhbz#1983053
- Rename %%_pyproject_ghost_distinfo and %%_pyproject_record to indicate they are private
- Automatically detect LICENSE files and mark them with %%license macro
@ -297,36 +389,34 @@ export HOSTNAME="rpmbuild" # to speedup tox in network-less mock, see rhbz#1856
* Fri Jul 09 2021 Python Maint <python-maint@redhat.com> - 0-44
- Escape weird paths generated by %%pyproject_save_files
- Fixes rhbz#1976363
- Support x.* versions in %%pyproject_buildrequires
- Fixes rhbz#1981558
- %%pyproject_buildrequires fallbacks to setuptools only if setup.py exists
- Fixes: rhbz#1976459
- Explicitly require the "basic" Python RPM macros
- Rebuilt for IMA sigs, glibc 2.34, aarch64 flags
Related: rhbz#1991688
* Thu Jul 01 2021 Tomas Hrnciar <thrnciar@redhat.com> - 0-43
- Generate BuildRequires from file
- Fixes: rhbz#1936448
* Tue Jun 29 2021 Miro Hrončok <mhroncok@redhat.com> - 0-42
- Don't accidentally treat "~= X.0" requirement as "~= X"
- Fixes rhzb#1977060
- Fixes rhbz#1977060
* Mon Jun 28 2021 Miro Hrončok <mhroncok@redhat.com> - 0-41
- Don't leak %%{_pyproject_builddir} to pytest collection
- Fixes rhbz#1935212
* Thu May 27 2021 Miro Hrončok <mhroncok@redhat.com> - 0-40
- Don't leak $TMPDIR outside of pyproject macros
- Set %%_pyproject_wheeldir and %%_pyproject_builddir relative to the source tree, not $PWD
* Thu Apr 22 2021 Miro Hrončok <mhroncok@redhat.com> - 0-39.2
* Mon Mar 29 2021 Miro Hrončok <mhroncok@redhat.com> - 0-39
- Handle tox provision (tox.requires / tox.minversion)
- Fixes: rhbz#1922495
- Generate BuildRequires on extras in lower case
- Fixes: rhbz#1947074
* Fri Apr 16 2021 Mohan Boddu <mboddu@redhat.com> - 0-39.1
- Rebuilt for RHEL 9 BETA on Apr 15th 2021. Related: rhbz#1947937
* Mon Mar 08 2021 Charalampos Stratakis <cstratak@redhat.com> - 0-38.1
- Disable tests on RHEL9 to remove tox dependency
- Fixes: rhbz#1937944
* Sun Feb 07 2021 Miro Hrončok <mhroncok@redhat.com> - 0-38
- Include nested __pycache__ directories in %%pyproject_save_files

Loading…
Cancel
Save