aedev.commands

software development operation commands helpers

Hint

this module is designed to provide a comprehensive set of constants, types, and helper functions for executing and managing DevOps on your Python projects in your Python virtual environments.

fundamental shell execution helpers

the helper functions run_git_traced() provides fundamental Git command execution with optional tracing/logging/debugging, error checking and exiting/quitting.

the logging of executed command lines and their console output is highly useful for debugging and protocolling purposes. this portion provides the following helper functions to implement logging for external commands. logging gets automatically enabled, if the corresponding log file exists.

sh_log(): writes a single command execution log entry.
sh_logs(): determines the file paths of the currently existing/enabled log files.
SHELL_LOG_FILE_NAME_SUFFIX: the default filename suffix for shell command log files.

Hint

this logging is implemented in run_git_traced() for all the git command execution helpers (git_*()) of this portion. to enable the extended logging of all executed git commands simply create a log file with the name git_sh.log, situated in the current working directory and/or in the users home directory (~).

git command helpers

this portion is providing helper functions for lots of git commands, most of them allow to activate git trace for debugging or intensive testing.

git commands will be executed in repo root folder of a project. the current working directory will be changed accordingly before a git command get executed (and restored to the old working directory after the execution).

to support GIT HOOKS that execute Python code, the Python virtual environment of a project will be activated before the git command get executed, and restored to the old value after the git command has finished.

extensive debugging using the GIT TRACE feature of the git commands will be activating if your app based on ConsoleApp got executed with the –debug_level/-D option specified.

bytes_file_diff(): returns the differences between a byte buffer and a file, using the git diff command.
check_commit_msg_file(): checks for the existence of a Git commit message file.
editable_project_root_path(): determine the project path of a project package installed as editable.
git_add(): executes the git add command to stage changes.
git_any(): executes any generic Git command.
git_branches(): determines branch names in a Git repository.
git_branch_files(): finds added, changed, or deleted files on a specified branch.
git_branch_remotes(): return the remote names where the specified branch name exists.
git_checkout(): executes the git checkout command to switch branches.
git_clone(): clone a git remote repository from a repository hoster onto your local machine.
git_commit(): executes the git commit command.
git_current_branch(): determine the currently checked-out branch of the specified git repository/project.
git_diff(): determine the uncommited/unstaged changes of the specified git repository/project.
git_fetch(): executes the git fetch command.
git_init_if_needed(): check and create a git repository if already not exists in the specified project.
git_merge(): merge current worktree with the specified [remote/]branch (exit app if an error occurred).
git_push(): executes the git push command.
git_ref_in_branch(): check if branch/tag/ref is in the specified branch.
git_remote_domain_group(): determine the domain and the repository owner group-/user-name from .git/config.
git_remotes(): retrieves the remote URLs of the repository.
git_renew_remotes(): renew the origin remote and optionally (if repo is forked) also the upstream remote.
git_status(): get the status of the project repository.
git_tag_add(): add a new tag onto the project in the specified project root path.
git_tag_list(): determine a list of matching tags of a local or remote git repository.
git_tag_remotes(): return the remote names where the specified tag name exists.
git_uncommitted(): lists all uncommitted files.
git_user_email(): retrieves the Git user’s email.
git_user_name(): retrieves the Git user’s name.
git_version_tag(): determines the current version tag of the repository.
owner_project_from_url(): determine the project path, including owner and project name from the remote url.

git command constants and types

EXEC_GIT_ERR_PREFIX: the prefix used to mark Git execution errors.
GIT_CLONE_CACHE_CONTEXT: the temporary folder context identifier used for Git clone downloads.
GIT_FOLDER_NAME: the default name of the Git-internal subfolder.
GIT_REMOTE_ORIGIN: the default name for the origin remote.
GIT_REMOTE_UPSTREAM: the default name for the upstream remote.
GIT_RELEASE_REF_PREFIX: the default prefix, used for release branches.
GIT_VERSION_TAG_PREFIX: the default prefix, used for version tags.
GitRemotesType: the type hint for a dictionary of Git remotes.

pip command helpers

pip_install(): install/check/find outdated external Python projects/distributions required by a Python project.
PIP_EDITABLE_PROJECT_PATH_PREFIX: displayed by the pip show command, used to parse editable project path.

virtual environment helpers

these helper functions are provided to assist with the management of Python virtual environments.

in_prj_dir_venv(): a context manager that temporarily changes the working directory and activates the project’s virtual environment.
in_venv(): set the VENV OS environment variables in os.environ via a patched dict within the context.
venv_bin_path(): determines the bin/scripts path of a Python virtual environment.
venv_check_prefixes(): determine the possible VENV root/prefix paths of all the supported VENV systems.
venv_module_var_val(): determine a variable value declared in a Python module, installed in any other VENV.
venv_name(): determine the name/id of the current VENV used by a project.
venv_os_env_vars(): returns the OS environment variables of a VENV (Python virtual environment).
venv_run_cmd_args(): determine the run shell command function args to set/switch the OS-env vars of a VENV.

the following example is executing in project root the shell/console command: pip install -r requirements.txt in order to install the required packages of a project into its local virtual environment by using the in_prj_dir_venv() context manager together with the shell command execution function run_cmd() and the constant PIP_CMD (which differs depending on your OS):

with in_prj_dir_venv(project_root_path):
    sh_err = run_cmd(PIP_CMD, "install", "-r", "requirements.txt")

Module Attributes

`EXEC_GIT_ERR_PREFIX`	for `run_git_traced()` to mark error in 1st output line
`GIT_CLONE_CACHE_CONTEXT`	temp directory context for git clone downloads
`GIT_FOLDER_NAME`	git subfolder in project path root of local repository
`GIT_REMOTE_ORIGIN`	git origin remote name of (fork) repository in user account
`GIT_REMOTE_UPSTREAM`	git upstream remote name of original/forked repository
`GIT_RELEASE_REF_PREFIX`	git repository release branch name prefix
`GIT_VERSION_TAG_PREFIX`	git repository version tag prefix
`PIP_EDITABLE_PROJECT_PATH_PREFIX`	caption/field-name of the console output of the pip show command.
`SHELL_LOG_FILE_NAME_SUFFIX`	default file name (suffix) of the shell log file
`VENV_SUB_DIR_NAMES`	recognized folder names of a project-local VENV
`VENV_ID_SEARCH_DEPTH`	number of folders to climb up to find a projects VENV name/id
`GitRemotesType`	git remote urls dict with keys like 'origin' / 'upstream'

Functions

`bytes_file_diff`(file_content, file_path[, ...])	return the differences between the content of a file against the specified file content buffer.
`check_commit_msg_file`(project_path, *hint_args)	check if the commit message file exists and if yes return the path of it.
`editable_project_root_path`(project_name)	determine the project path of a project package installed as editable.
`git_add`(project_path, *extra_args)	execute the git add command.
`git_any`(project_path, *args)	execute any git command.
`git_branch_files`(project_path[, ...])	find all added/changed/deleted/renamed/unstaged worktree files that are not merged into the main branch.
`git_branch_remotes`(project_path, branch_pattern)	return the remote names where the specified branch name exists.
`git_branches`(project_path, *extra_args)	determine all branch names with the git branch command.
`git_checkout`(project_path, *extra_args[, ...])	checkout git branch.
`git_clone`(repo_parent_url, project_name, ...)	clone a git remote repository from a repository hoster onto your local machine.
`git_commit`(project_path, project_version, ...)	execute the command 'git commit' for the specified project.
`git_current_branch`(project_path)	determine the currently checked-out branch of the specified git repository/project.
`git_diff`(project_path, *extra_args)	determine the uncommited/unstaged changes of the specified git repository/project.
`git_fetch`(project_path, *extra_args[, ...])	fetch repository from remotes.
`git_init_if_needed`(project_path[, author, ...])	check if a git repository already exists in the specified project root folder, and create it if not.
`git_merge`(project_path, from_branch, ...[, ...])	merge current worktree with the specified [remote/]branch (exit app if an error occurred).
`git_push`(project_path, remote_repo_url, ...)	push the repo of the specified project with the specified branch/tags to the specified remote.
`git_ref_in_branch`(project_path, ref[, branch])	check if branch/tag/ref is in the specified branch.
`git_remote_domain_group`(project_path[, ...])	determine the domain and the repository owner group-/user-name from the git remote configuration (.git/config).
`git_remotes`(project_path)	get mapping of hte project repository remotes.
`git_renew_remotes`(project_path, origin_url)	renew the origin remote and optionally (if repo is forked) also the upstream remote.
`git_status`(project_path[, verbose])	get the status of the project repository.
`git_tag_add`(project_path, tag[, ...])	add a new tag onto the project in the specified project root path.
`git_tag_list`(project_path[, remote, tag_pattern])	determine a list of matching tags of a local or remote git repository.
`git_tag_remotes`(project_path, tag_pattern[, ...])	return the remote names where the specified tag name exists.
`git_uncommitted`(project_path)	determine the changed/untracked/uncommitted files of a git repository/project.
`in_prj_dir_venv`([project_path, venv_id])	set CWD to the project root, os.environ from .env files and the specified or .python-version-configured VENV.
`in_venv`([venv_id])	set the VENV OS environment variables in `os.environ` via a patched dict within the context.
`owner_project_from_url`(remote_url)	determine the owner and project name path from the specified git remote repository url.
`pip_install`(project_path, *required_projects)	install or check/find outdated external Python projects/distributions required by a Python project.
`run_git_traced`(err_code, *cmd_args[, ...])	run git command with optional git trace output, returning the stdout lines cleaned from any git trace messages.
`sh_log`(*cmd_args[, cl_err, output_lines, ...])	append a log entry to each existing/enabled shell command log file.
`sh_logs`([log_enable_dir, log_name_prefix])	determine paths of the existing/enabled shell command log files, optionally enabling/creating a new log file.
`venv_bin_path`(venv_id)	determine the absolute bin/executables folder path of the specified, local or active Python virtual environment.
`venv_check_prefixes`(venv_id)	determine the possible VENV root/prefix paths of all the supported VENV systems.
`venv_module_var_val`(import_name, var_name[, ...])	determine a variable value that is declared in a Python module in any other VENV.
`venv_name`([project_path])	determine the name/id of the current VENV used by a project.
`venv_os_env_vars`(venv_id[, app_obj])	return the OS env vars of the Python virtual environment specified by their VENV id.
`venv_run_cmd_args`(bin_path, app_obj)	determine the args to call one of the shell command runner functions to set OS-env vars of a VENV.

EXEC_GIT_ERR_PREFIX = 'run_cmd() returned error ': for run_git_traced() to mark error in 1st output line

GIT_CLONE_CACHE_CONTEXT = 'shell.git_clone': temp directory context for git clone downloads

GIT_FOLDER_NAME = '.git': git subfolder in project path root of local repository

GIT_REMOTE_ORIGIN = 'origin': git origin remote name of (fork) repository in user account

GIT_REMOTE_UPSTREAM = 'upstream': git upstream remote name of original/forked repository

GIT_RELEASE_REF_PREFIX = 'release': git repository release branch name prefix

GIT_VERSION_TAG_PREFIX = 'v': git repository version tag prefix

PIP_EDITABLE_PROJECT_PATH_PREFIX = 'Editable project location: ': caption/field-name of the console output of the pip show command.

SHELL_LOG_FILE_NAME_SUFFIX = '_sh.log': default file name (suffix) of the shell log file

VENV_SUB_DIR_NAMES = ('.venv/', 'venv/', 'env/', '.env/'): recognized folder names of a project-local VENV

VENV_ID_SEARCH_DEPTH = 3: number of folders to climb up to find a projects VENV name/id

GitRemotesType

git remote urls dict with keys like ‘origin’ / ‘upstream’

alias of dict[str, str]

bytes_file_diff(file_content, file_path, line_sep='\\n')[source]

return the differences between the content of a file against the specified file content buffer.

Parameters:

file_content¶ (bytes) – older file bytes to be compared against the file content of the file specified by the file_path argument.
file_path¶ (str) – path to the file of which newer content gets compared against the file bytes specified by the file_content argument.
line_sep¶ (str) – string used to prefix, separate and indent the lines in the returned output string.

Return type:

str

Returns:

differences between the two file contents, compiled with the git diff command.

check_commit_msg_file(project_path, *hint_args, commit_msg_file='.commit_msg.txt')[source]

check if the commit message file exists and if yes return the path of it.

Parameters:

project_path¶ (str) – project root path.
hint_args¶ – hint arguments.
commit_msg_file¶ (str) – name of the git commit message file (default=:data:aedev.base.COMMIT_MSG_FILE_NAME).

Return type:

str

Returns:

the path of the git commit message file of this project.

Raises:

FileNotFoundError|shutdown if the commit message file is not readable or does not exist.

editable_project_root_path(project_name)[source]

determine the project path of a project package installed as editable.

Parameters:: project_name¶ (str) – project|package name to search for.
Return type:: str
Returns:: project source root path of an editable installed package or empty string, if the package is not installed as editable.

git_add(project_path, *extra_args)[source]

execute the git add command.

Parameters:

project_path¶ (str) – project path.
extra_args¶ (str) – additional arguments passed onto git add command. default=[“-A”].

git_any(project_path, *args)[source]

execute any git command.

Parameters:

project_path¶ (str) – path to project root folder.
args¶ (str) – arguments passed onto the git executable. first arg is the “git” command.

Return type:

list[str]

Returns:

list of console output lines of the git command optionally including the exit error code (marked with EXEC_GIT_ERR_PREFIX in the first returned list item), like returned by run_git_traced().

git_branches(project_path, *extra_args)[source]

determine all branch names with the git branch command.

Parameters:

project_path¶ (str) – path to project root folder.
extra_args¶ (str) – additional arguments passed onto git branch command. default=(“-a”, “–no-color”).

Return type:

list[str]

Returns:

list of branch names of the project repo.

git_branch_files(project_path, branch_or_tag='develop', untracked=False, skip_file_path=<function <lambda>>)[source]

find all added/changed/deleted/renamed/unstaged worktree files that are not merged into the main branch.

Parameters:

project_path¶ (str) – path of the project root folder. pass empty string to use the current working directory.
branch_or_tag¶ (str) – branch(es)/tag(s)/commit(s) passed to git diff to specify the changed files between version(s).
skip_file_path¶ (Callable[[str], bool]) – called for each found file passing the file path relative to the project root folder (specified by the project_path argument), returning True to exclude/skip the file with passed file path.
untracked¶ (bool) – pass True to include untracked files from the returned result set.

Return type:

set[str]

Returns:

set of file paths relative to worktree root specified by the project root path specified by the project_path argument.

Hint

see also func:git_uncommitted and the unit tests for the differences between them.

git_branch_remotes(project_path, branch_pattern, remote_names=())[source]

return the remote names where the specified branch name exists.

Parameters:

project_path¶ (str) – path of the project root folder.
branch_pattern¶ (str) – branch name pattern to search for.
remote_names¶ (Iterable[str]) – iterable with the remote names. determined with git_remotes() if not specified.

Return type:

list[str]

Returns:

list of remote names where the branch exists.

git_checkout(project_path, *extra_args, new_branch='', exit_on_err=True, force=False, remote_names=())[source]

checkout git branch.

Parameters:

project_path¶ (str) – path of the project root folder.
extra_args¶ (str) – additional arguments passed onto git checkout command.
new_branch¶ (str) – new branch name to create and check out.
exit_on_err¶ (bool) – specify False to not exit the Python app on any git checkout error.
force¶ (bool) – pass True to ignore uncommitted files and if undefined branch.
remote_names¶ (Iterable[str]) – iterable with the remote names. determined with git_remotes() if not specified.

Return type:

str

Returns:

error message or empty string if no error occurred.

git_clone(repo_parent_url, project_name, *extra_args, branch_or_tag='', parent_path='', enable_log=False)[source]

clone a git remote repository from a repository hoster onto your local machine.

Parameters:

repo_parent_url¶ (str) – repository parent/group url (without the project name to clone).
project_name¶ (str) – project name to clone.
extra_args¶ (str) – extra arguments passed onto the git clone command.
branch_or_tag¶ (str) – repo branch to clone. if not specified then the main branch will be cloned.
parent_path¶ (str) – destination path on the local machine to clone onto. if not specified a temporary folder will be used.
enable_log¶ (bool) – pass True to enable git shell command logging.

Return type:

str

Returns:

path to the cloned repository folder or empty string if an error occurred.

Hint

there could occur a user/password prompt if repo is private/invalid!

git_commit(project_path, project_version, *extra_args, commit_msg_text='', commit_msg_file='.commit_msg.txt')[source]

execute the command ‘git commit’ for the specified project.

Parameters:

project_path¶ (str) – path of the project root folder, in which this git command gets executed.
project_version¶ (str) – project version string.
extra_args¶ (str) – additional options or args passed to the git commit command line, e.g., [”–patch”, “–dry-run”]. except from the –file option, which will be added by this function with the name of the git commit message file.
commit_msg_text¶ (str) – used commit message. if specified then the argument in commit_msg_file will be ignored.
commit_msg_file¶ (str) – name of the git commit message file (default=:data:aedev.base.COMMIT_MSG_FILE_NAME).

git_current_branch(project_path)[source]

determine the currently checked-out branch of the specified git repository/project.

Parameters:: project_path¶ (str) – project root folder of the git repo.
Return type:: str
Returns:: name of the current branch, or an empty string if no branch is checked out.

git_diff(project_path, *extra_args)[source]

determine the uncommited/unstaged changes of the specified git repository/project.

Parameters:

project_path¶ (str) – project root folder.
extra_args¶ (str) – additional options and refs passed onto the git diff command, apart from the always added options: –no-color, –find-copies-harder, –find-renames and –full-index. pass e.g. –compact-summary or –name-only for a more compact output/return.

Return type:

list[str]

Returns:

list of console output lines of the git diff command, optionally including the exit error code (marked with EXEC_GIT_ERR_PREFIX in the first returned list item), like returned by run_git_traced().

git_fetch(project_path, *extra_args, exit_on_err=False)[source]

fetch repository from remotes.

Parameters:

project_path¶ (str) – project root folder.
extra_args¶ (str) – additional options and arguments (remote name) for the git fetch command.
exit_on_err¶ (bool) – specify True to exit the Python app on any git fetch error.

Return type:

list[str]

Returns:

list of lines from the console output that record an error (e.g. if no .git folder).

git_init_if_needed(project_path, author='', email='', main_branch='develop')[source]

check if a git repository already exists in the specified project root folder, and create it if not.

Parameters:

project_path¶ (str) – project root path
author¶ (str) – author of the project/repo added to git config (if specified).
email¶ (str) – email address of the author added to git config (if specified).
main_branch¶ (str) – first main branch name to be checked out if repo did not exist / got created. if not specified then the value of aedev.base.DEF_MAIN_BRANCH is used. pass empty string to not do the initial checkout on a not existing repository.

Return type:

bool

Returns:

boolean True if a new repo got created and initialized, else False.

git_merge(project_path, from_branch, *extra_options, commit_msg_text='', commit_msg_file='.commit_msg.txt', exit_on_err=False)[source]

merge current worktree with the specified [remote/]branch (exit app if an error occurred).

Parameters:

project_path¶ (str) – project root path of worktree to merge.
from_branch¶ (str) – branch (or commit/tag) to merge into current worktree with (optional with a leading upstream/origin remote name, e.g. “upstream/main_branch”).
extra_options¶ (str) – extra arguments for the git command
commit_msg_text¶ (str) – commit message used for the optional merge commit. if specified then the argument in commit_msg_file will be ignored.
commit_msg_file¶ (str) – name of the git commit message file (default=:data:aedev.base.COMMIT_MSG_FILE_NAME).
exit_on_err¶ (bool) – specify True to exit the Python app on any git push error.

Return type:

list[str]

Returns:

list with output lines of the git merge command (like returned by the function run_git_traced(), used to execute this git command). if the git command returned with an error code and the argument in exit_on_err got not specified or as a True argument, then the app will quit. if exit_on_err got specified as ‘False’ and git push returned an error code, then it will be returned in the first line/list-item (prefixed with EXEC_GIT_ERR_PREFIX).

git_push(project_path, remote_repo_url, *options_and_refs, exit_on_err=True)[source]

push the repo of the specified project with the specified branch/tags to the specified remote.

Parameters:

project_path¶ (str) – project root folder.
remote_repo_url¶ (str) – remote repo url. if git credentials are not configured on the local system, then a user token or password has to be included for the authentication against remote.
options_and_refs¶ (str) – extra arguments for the git command. pass additional options, e.g. “–delete” or “–set-upstream” or “-u”, and any references like branch/tag names to be pushed.
exit_on_err¶ (bool) – specify False to not exit the Python app on any git push error.

Return type:

list[str]

Returns:

list with output lines of the git push command (like returned by the function run_git_traced(), used to execute this git push command). if git push returned with an error code and the argument in exit_on_err got not specified or as a True argument, then the app will quit. if exit_on_err got specified as ‘False’ and git push returned an error code, then it will be returned in the first line/list-item (prefixed with EXEC_GIT_ERR_PREFIX).

git_ref_in_branch(project_path, ref, branch='origin/develop')[source]

check if branch/tag/ref is in the specified branch.

Parameters:

project_path¶ (str) – project worktree root path.
ref¶ (str) – any ref like a tag or another branch, to be searched within branch.
branch¶ (str) – branch to be searched in for ref. if not specified then it defaults to the remote/origin main branch.

Return type:

bool

Returns:

boolean True if the ref got found in the branch, else False.

git_remote_domain_group(project_path, origin_name='origin', upstream_name='upstream', remote_urls=None)[source]

determine the domain and the repository owner group-/user-name from the git remote configuration (.git/config).

Parameters:

project_path¶ (str) – path to the project root folder of the repository.
origin_name¶ (str) – name of the origin git remote (to push to).
upstream_name¶ (str) – name of the upstream git remote (to request a merge from).
remote_urls¶ (dict[str, str] | None) – remote urls. defaults to git_remotes() if not provided.

Return type:

tuple[str, str]

Returns:

tuple with the domain and the user/group of the upstream repository url. if no upstream url is set/configured (not forked) then the origin url is used. if there is no remote configured at all, then the returned tuple contains two empty strings.

git_remotes(project_path)[source]

get mapping of hte project repository remotes.

Parameters:: project_path¶ (str) – path to the project root folder of the repository.
Return type:: dict[str, str]
Returns:: dict with the remote ids as keys and the url as the values.

git_renew_remotes(project_path, origin_url, upstream_url='', origin_name='origin', upstream_name='upstream', remotes=None)[source]

renew the origin remote and optionally (if repo is forked) also the upstream remote.

Parameters:

project_path¶ (str) – project root folder.
origin_url¶ (str) – new url of the origin repo. will append a missing .git extension to the specified url.
upstream_url¶ (str) – new url of the upstream repo if forked. if specified and differs to the actual upstream url and to the specified origin url, then it will be automatically added. appends a missing .git extension to the specified url.
origin_name¶ (str) – name of the origin git remote (to push to). if not specified defaults to GIT_REMOTE_ORIGIN.
upstream_name¶ (str) – name of the upstream git remote (to request a merge from). if not specified defaults to GIT_REMOTE_UPSTREAM.
remotes¶ (dict[str, str] | None) – pass the actual git remotes to prevent multiple execution of the git remote command. determined via git_remotes() if not specified.

Return type:

list[str]

Returns:

list of console output lines of the executed git remote commands. an empty list no errors/warnings got outputted by the git remote command.

git_status(project_path, verbose=False)[source]

get the status of the project repository.

Parameters:

project_path¶ (str) – project root path.
verbose¶ (bool) – pass True to return a more verbose console output (using –branch -vv –porcelain=2).

Return type:

list[str]

Returns:

console output of the git status command.

git_tag_add(project_path, tag, commit_msg_text='', commit_msg_file='.commit_msg.txt')[source]

add a new tag onto the project in the specified project root path.

Parameters:

project_path¶ (str) – project root path.
tag¶ (str) – tag to add.
commit_msg_text¶ (str) – commit message used for the optional merge commit. if specified then the argument in commit_msg_file will be ignored.
commit_msg_file¶ (str) – name of the git commit message file (def=COMMIT_MSG_FILE_NAME).

Return type:

list[str]

Returns:

console output of the git tag –annotate command.

git_tag_list(project_path, remote='', tag_pattern='*')[source]

determine a list of matching tags of a local or remote git repository.

Parameters:

project_path¶ (str) – project root folder.
remote¶ – name of a remote. if not specified then only local tags will be determined.
tag_pattern¶ (str) – matching pattern. if not specified then all tags will be returned.

Return type:

list[str]

Returns:

list of matching repo tags, ordered via the git ls-remote option –sort=version:refname. an empty list will be returned if no tag is matching the specified tag pattern or an error occurred or if the project has no .git folder.

git_tag_remotes(project_path, tag_pattern, remote_names=())[source]

return the remote names where the specified tag name exists.

Parameters:

project_path¶ (str) – path of the project root folder.
tag_pattern¶ (str) – tag pattern to search for.
remote_names¶ (Iterable[str]) – iterable with the remote names. determined with git_remotes() if not specified.

Return type:

list[str]

Returns:

list of remote names where the tag exists.

git_uncommitted(project_path)[source]

determine the changed/untracked/uncommitted files of a git repository/project.

Parameters:: project_path¶ (str) – project root folder.
Return type:: set[str]
Returns:: set of changed/untracked/uncommitted file names. an empty set will be returned if the git repo is not initialized or if there are no uncommitted files in the git repo specified by project_path.

Hint

see also func:git_branch_files and the unit tests for the differences between them.

in_prj_dir_venv(project_path='.', venv_id='')[source]

set CWD to the project root, os.environ from .env files and the specified or .python-version-configured VENV.

Parameters:

project_path¶ (str) – path to the project root folder to switch the current working directory in this context. using the actual CWD if not specified.
venv_id¶ (str) – name/id of the Python Virtual Environment to activate in this context. if not specified (or as empty string), then the VENV configured via the file .python-version will be activated.

Return type:

Iterator[None]

Returns:

in_venv(venv_id='')[source]

set the VENV OS environment variables in os.environ via a patched dict within the context.

Parameters:: venv_id¶ (str) – the name of the VENV to activate. if not specified, then the VENV of the project in the current working directory tree will be activated.
Return type:: Iterator[None]

Note

changes done within this context on any variable value in os.environ will have no effect on the real/underlying environment variable value of the OS.

owner_project_from_url(remote_url)[source]

determine the owner and project name path from the specified git remote repository url.

Parameters:: remote_url¶ (str) – remote repository url.
Return type:: str
Returns:: the owner name and the project name, separated with a slash character.

pip_install(project_path, *required_projects, cooldown_period='', dry_run=False, force_reinstall=False, return_implicits=False)[source]

install or check/find outdated external Python projects/distributions required by a Python project.

Parameters:

project_path¶ (str) – path to the project root folder for which the requirements will get installed/checked.
required_projects¶ (str) – list of required project names with optional project version numbers (separated by one of the comparison operators supported by pip, like e.g. PROJECT_VERSION_SEP).
cooldown_period¶ (str) – either ISO 8601 datetime (e.g., ‘2023-01-01T00:00:00Z’) or number of days (e.g., ‘P6D’ for uploaded at least 6 days ago).
dry_run¶ (bool) – pass True to only check which (outdated) external Python projects would get installed.
force_reinstall¶ (bool) – pass True to force the reinstallation of the required/outdated project versions.
return_implicits¶ (bool) – pass True to return also implicit installed Python projects as comments.

Return type:

dict[str, dict[str, bool | str]]

Returns:

dict of installed/outdated projects, with their normalized pip name as keys and another/inner dict as values. the inner dict provides a “version” key with the installed/found version as value. if True got specified for the return_implicits argument, then also a “requested” key will be available in the inner dict with a boolean value, which is True for explicit installed/requiered projects and False for implicit ones.

run_git_traced(err_code, *cmd_args, exit_on_err=False, log_enable_dir='')[source]

run git command with optional git trace output, returning the stdout lines cleaned from any git trace messages.

Parameters:

err_code¶ (int) – error code to be passed onto the console as exit code if the git command set an error code and the exit_on_err argument is either True or a nonempty string.
cmd_args¶ (str) – command line string or a sequence of command name and separate line arguments to be run/executed on the console/shell.
exit_on_err¶ (bool) – pass True to exit the app on error.
log_enable_dir¶ (str) – pass the path of the directory in which git shell command logging have to get enabled.

Return type:

list[str]

Returns:

output lines of git command - cleaned from GIT_TRACE messages, if exit_on_err got specified as ‘False’ and the executed git command returned an error code, then the error code will be returned in the first line/list-item (prefixed with EXEC_GIT_ERR_PREFIX).

Raises:

AssertationError if first argument of cmd_args is not “git”.

sh_log(*cmd_args, cl_err=0, output_lines=(), log_file_paths=(), log_name_prefix='')[source]

append a log entry to each existing/enabled shell command log file.

Parameters:

cmd_args¶ (str) – command line arguments or comment line (if cmd_args[0] starts with a # character).
cl_err¶ (int) – command exit code.
output_lines¶ (Iterable[str]) – console output lines.
log_file_paths¶ (Iterable[str]) – log file paths - if specified then the search of the default locations for log files will be skipped and therefore log_name_prefix will be ignored.
log_name_prefix¶ (str) – log file name prefix. extended with the SHELL_LOG_FILE_NAME_SUFFIX results in the file name to search for (and to log into if exists).

sh_logs(log_enable_dir='', log_name_prefix='')[source]

determine paths of the existing/enabled shell command log files, optionally enabling/creating a new log file.

Parameters:

log_enable_dir¶ (str) – specify the directory/folder path in which to enable shell command logging, by creating a new log file (and the folder) if they not exist. valid folder paths are either the CWD where the shell command get executed (e.g. the project root folder) or the users home directory (~).
log_name_prefix¶ (str) – log file name prefix. extended with the SHELL_LOG_FILE_NAME_SUFFIX results in the file name to search for (and to log into if exists).

Return type:

list[str]

Returns:

list of existing/enabled shell log file name paths.

venv_bin_path(venv_id)[source]

determine the absolute bin/executables folder path of the specified, local or active Python virtual environment.

Parameters:

venv_id¶ (str) – name/id of the VENV or the name of a project-local VENV sub folder (with a trailing slash).

Return type:

str

Returns:

absolute path of the bin/Scripts folder of the specified/determined virtual environment or an empty string if no VENV could be found with the specified/determined id/name.

Note

under Windows/win32 the base name of the returned path is ‘Scripts’ (not ‘bin’), and some executables may have a file extension (e.g., activate.bat and python.exe). ensures “/” path separators to work properly in WSL/bash-emulation under MS Windows.

venv_check_prefixes(venv_id)[source]

determine the possible VENV root/prefix paths of all the supported VENV systems.

Parameters:: venv_id¶ (str) – name/id of the VENV or the name of a project-local VENV sub folder (with a trailing slash).
Return type:: list[tuple[str, ...]]
Returns:: list of joinable path parts of the possible and supported VENV root/prefix paths.

venv_module_var_val(import_name, var_name, cwd='.', venv_id='', validator=<function <lambda>>)[source]

determine a variable value that is declared in a Python module in any other VENV.

Parameters:

import_name¶ (str) – import-/dot-name of the module to get the variable value from.
var_name¶ (str) – name of the variable declared within the module. the value of the variable has to be parsable by ast.literal_eval().
cwd¶ (str) – the CWD to set for the Python interpreter of the VENV to run.
venv_id¶ (str) – the name/id of the Python virtual environment to use. using the VENV of the current working directory or cwd, if not specified.
validator¶ (Callable[[Any], bool]) – callable called for each line of the console output, with the variable value as argument, returning True if the variable value is correct. useful if command line output may contain extra prefixes/lines (e.g. warnings), that have to be skipped/ignored.

Return type:

Any | UnsetType | None

Returns:

module variable value (parseable by ast.literal_eval()), or UNSET on error, if the module got not found or if the module variable doesn’t exist.

Note

the PyPI package/project ae.system has to be installed in the used/destination VENV.

venv_name(project_path='.')[source]

determine the name/id of the current VENV used by a project.

Parameters:: project_path¶ (str) – path to the project root folder; assuming and using the actual CWD if not specified.
Return type:: str
Returns:: the determined VENV name/id or an empty string if no VENV got found or is active. the VENV name/id will be determined either from the first line of a local .python-version file, or from an existing and allowed subfolder name (VENV_SUB_DIR_NAMES contains the commonly used ones) of a project-local VENV. the search starts in the specified project path (or the CWD) and goes up until 2 parent directories above it. if it did not find any there, then the name of the currently active VENV (determined via the function ae.system.os_env_venv()) will be returned.

venv_os_env_vars(venv_id, app_obj=None)[source]

return the OS env vars of the Python virtual environment specified by their VENV id.

Parameters:

venv_id¶ (str) – the id/name of the Python virtual environment (VENV) get the OS env vars for.
app_obj¶ (ConsoleApp | UnsetType | None) – optional ConsoleApp instance, used for logging/console output.

Return type:

dict[str, str]

Returns:

dict with the OS environment variable values of the specified or currently set VENV or an empty dict if the requested or no VENV was active, or if VENV is not supported.

venv_run_cmd_args(bin_path, app_obj)[source]

determine the args to call one of the shell command runner functions to set OS-env vars of a VENV.

Parameters:

bin_path¶ (str) – the bin/Scripts folder path of the VENV.
app_obj¶ (AppBase | UnsetType | None) – optional AppBase instance, used for logging/console output.

Return type:

tuple[list[str], dict[str, Any]]

Returns:

OS- and VENV-specific command line args to be passed to run_logged_cmd() (or run_cmd()) in order to set VENV-specific OS-environment variables.