aedev.project_manager.__main__
Python Project Manager
constants, classes and helper functions
this main module of the project-manager tool implements the CLI including the supported dev-ops (actions)
the classes GithubCom and GitlabCom are providing access to the related
repository hosting services and the PythonanywhereCom to check and deploy web apps.
helper functions like web_app_version() determining the remote version of a deployed django app project package.
constants and types are declared in this module and in the project-internal utils,
like e.g. PPF and pprint providing
consistent indented console outputs via pretty print formatters.
external helpers dependencies
helper functions to categorize and maintain project attributes are provided by the two external
module aedev.commands and aedev.project_vars, portions of the aedev namespace.
helper functions provided by the aedev.commands encapsulate git and pip commands.
increment_version(), latest_remote_version() or
replace_file_version() are used by this module to determine or manipulate
the local/remote/released versions of your projects.
the ae namespace portion ae.managed_files provides the base functionality
(e.g., deploy_template() and patch_string()),
extended by the helpers provided by the project-internal module templates,
to generate managed project files from the aedev.*_tpls template projects
(like e.g. aedev.project_tpls or aedev.app_tpls). example implementations can be found .e.g.
in the add_file() action and in the functions check_templates()
and deploy_template().
the portion ae.pythonanywhere portion encapsulates the web API to deploy Django web applications
to the Pythonanywhere webserver.
Module Attributes
main app instance of this pjm tool, initialized by |
Functions
|
add any file to the project working trees of parent/root and children/portions. |
|
add any file into the project working tree. |
|
build gui app with buildozer, add LIBS to make a clean/full build and EMBED to include APK to share. |
|
run integrity checks for the specified children of a parent or portions of a namespace. |
|
integrity check of files/folders completeness, managed/template files update-state, and CI tests. |
|
clone specified namespace-portion/parent-child repositories to the local machine. |
|
clone remote repo to the local machine. |
|
commit changes to children of a namespace/parent using the individually prepared commit message files. |
|
commit changes of a single project to the local repo using the prepared commit message file. |
|
delete a file or an empty folder from parent/root and children/portions project roots. |
|
delete a file or an empty folder from the project. |
initialize main app instance. |
|
|
install parent children or namespace portions as editable on the local machine. |
|
install the project as editable from the source/project root folder. |
|
main app script |
|
create or complete/renew a gui app project. |
|
initialize or renew parent folder children or namespace portions. |
|
create or complete/renew a django project. |
|
create or complete/renew a module project. |
|
create or complete/renew a namespace root package. |
|
create or complete/renew a package project. |
|
create or complete/renew a playground project. |
prepare and run app |
|
|
run code checks and prepare/overwrite the commit message file for a bulk-commit of children projects. |
|
run code checks and prepare/overwrite the commit message file for the commit of a single project/package. |
|
refresh frozen requirements and managed files from templates in all the children projects. |
|
refresh/renew all the *requirements_frozen.txt files and all the managed files of the specified project. . |
|
rename a file or folder in parent/root and children/portions projects. |
|
rename a file or folder in the project tree. |
|
complete/renew/update the local children projects of the specified parent/namespace-root. |
|
complete/renew/update an existing project. |
|
run console command for the specified portions/children of a namespace/parent. |
|
get available/registered/implemented actions info of the specified/current project and remote. |
|
show package versions (local, remote and on pypi) for the specified children of a namespace/parent. |
|
display package versions of worktree, remote repo(s), latest PyPI release and default app/web host. |
|
create or update a mirror of the actual repo onto the specified remote/host. |
|
upgrade project requirements|dependencies, optionally as editable package. |
|
determine the version of a deployed django project package. |
Classes
remote connection and actions on remote repo in gitHub.com. |
|
remote connection and actions on gitlab.com. |
|
remote actions on remote web host pythonanywhere.com (to be specified by --web_domain option). |
|
base class registering subclasses as remote repo or web host class in |
- cae = None
main app instance of this pjm tool, initialized by
init_main()
- _action(*project_types, **deco_kwargs)[source]
parametrized decorator to declare functions and
RemoteHostmethods as pjm actions.- Return type:
- _check_resources_i18n_ae(file_name, content)[source]
check a translation text file with ae_i18n portion message texts.
- _check_resources_i18n_po(file_name, content)[source]
check a translation text file with GNU gettext message texts.
- _check_version(version_number, prefix_to_check='')[source]
check project version, exit the app on any format error, and return the checked version without the prefix.
- Return type:
- _get_pdv(**kwargs)[source]
create a pdv instance from the specified kwargs, check it for errors and if it has errors then exit app.
- _init_act_args_check(ini_pdv, act_spec, act_name, act_args, act_flags)[source]
check and possibly complete the command line arguments and split optional action flags from action args.
called after _init_act_exec_args/INI_PDV-initialization.
- _init_act_exec_args()[source]
prepare execution of an action requested via command line arguments and options.
init project dev vars
checks if action is implemented
check action arguments
run optional pre_action.
- _init_children_pdv_args(ini_pdv, act_args)[source]
get package names of the portions specified as command line args, optionally filtered by –branch option.
- Return type:
- _show_status(ini_pdv)[source]
show git status and a guess of the next action for the specified/current project on the local machine.
- Return type:
- _update_project(ini_pdv, remote_names=(), hard_reset=False)[source]
update projects main branch from remotes, returning an empty string or a text block with error messages.
- Parameters:
- Return type:
- Returns:
list of errors. some errors get ignored and not returned.
- class RemoteHost[source]
Bases:
objectbase class registering subclasses as remote repo or web host class in
REGISTERED_HOSTS_CLASS_NAMES.- classmethod __init_subclass__(**kwargs)[source]
register a remote host class name; called on declaration of a subclass of
RemoteHost.
- repo_merge_src_dst_fork_branch(ini_pdv)[source]
determine instances of remote source and destination repositories, if it is forked and the branch to merge.
- repo_release_project(ini_pdv, version_tag)[source]
prepare project release and reset local repository, optionally create release branch and publish to PyPI.
- Parameters:
ini_pdv¶ (
ProjectDevVars) – project dev vars.
- Return type:
- Returns:
end-of-action confirmation message, to be printed to console.
- class GithubCom[source]
Bases:
RemoteHostremote connection and actions on remote repo in gitHub.com.
- connect(ini_pdv)[source]
connect to gitHub.com remote host.
- Parameters:
ini_pdv¶ (
ProjectDevVars) – project dev vars (only using the value of the ‘repo_token’ variable).- Return type:
- Returns:
boolean True on successful authentication else False.
- create_branch(group_repo, branch_name, tag_name)[source]
create a new remote branch onto/from the tag name.
- group_obj(user_or_org_name)[source]
instantiate am authenticated-user or organization object from the specified name.
:param _sphinx_paramlinks_aedev.project_manager.__main__.GithubCom.group_obj.user_or_org_name:name of a user or organization.
- repo_obj(err_code, err_msg, group_repo)[source]
convert user repo names to a repository instance of the remote api.
- Parameters:
err_code¶ (
int) – error code, pass 0 to not quit if a project is not found.err_msg¶ (
str) – error message to display on error. will be extended with the group and project names from thegroup_repoargument.group_repo¶ (
str) – string with owner-user-name/repo-name of the repository, e.g. “UserName/RepositoryName”.
- Return type:
Optional[Repository]- Returns:
GitHub repository if found, else return None if err_code is zero else quit.
- fork_project(ini_pdv, fork_repo_path)[source]
create/renew a fork of a remote repo specified via the 1st argument, into our user namespace.
- push_project(ini_pdv)[source]
push the current/specified branch of project/package version-tagged to the remote repository host.
- Parameters:
ini_pdv¶ (
ProjectDevVars) – project dev vars.
- release_project(ini_pdv, version_tag)[source]
update local main branch from origin, and if pip_name is set, then release the latest/specified version too.
- Parameters:
ini_pdv¶ (
ProjectDevVars) – project dev vars.version_tag¶ (
str) – push version tag in the formatv<version-number>to release orLATESTto use the version tag of the latest git repository version.
- class GitlabCom[source]
Bases:
RemoteHostremote connection and actions on gitlab.com.
- branch_merge_requests(ini_pdv, branch)[source]
determine the merge/pull requests (opened or closed) for the specified branch.
- Parameters:
ini_pdv¶ (
ProjectDevVars) – project dev vars.branch¶ (
str) – name of the branch to determine the merge/pull requests.
- Return type:
list[ProjectMergeRequest]- Returns:
found merge/pull requests for the specified branch or empty list on error.
- connect(ini_pdv)[source]
connect to gitlab.com remote host.
- Parameters:
ini_pdv¶ (
ProjectDevVars) – project dev vars (REPO_HOST_PROTOCOL, host_domain, host_token).- Return type:
- Returns:
boolean True on successful authentication else False.
- create_branch(owner_prj, branch_name, tag_name)[source]
create a new remote branch onto/from the tag name.
- init_new_repo(ini_pdv)[source]
create a remote group/user project specified in ini_pdv or quit with error if group/user not found.
- Parameters:
ini_pdv¶ (
ProjectDevVars) – project dev vars.- Return type:
- Returns:
error message or empty string if no errors occurred.
- merge_pushed_project(pdv, request=None, message='', max_wait=6.9)[source]
merge an MR of the specified project.
- Parameters:
pdv¶ (
ProjectDevVars) – project dev vars.request¶ (
Optional[ProjectMergeRequest]) – pass MergeRequest instance for direct merge of unforked repository.message¶ (
str) – commit message file content. will be read from project root folder if empty|not-passed.max_wait¶ (
float) – maximum waiting time in seconds for all the retries of the merge. the delay between each retry can be specified via the –delay option.
- Return type:
- Returns:
number of retries left. returns zero if merge did fail (consuming all retries).
- repo_obj(err_code, owner_project)[source]
create Project instance of a remote repository specified by its namespace path or its endswith-fragment.
- Parameters:
- Return type:
Optional[Project]- Returns:
python-gitlab project instance if found, else return None if err_code is zero else quit.
- project_owner(ini_pdv)[source]
get the owner (group|user) of the project specified by ini_pdv or quit with error if group/user not found.
- Parameters:
ini_pdv¶ (
ProjectDevVars) – project dev vars.- Return type:
Union[Group,User]- Returns:
instance of Group or User, determined via the user-/group-names specified by ini_pdv.
- clean_releases(ini_pdv)[source]
delete local+remote release tags and branches of the specified project that got not published to PYPI.
- fork_project(ini_pdv, owner_project_path)[source]
create or renew a fork of a remote repo, specified via the 1st argument, into our user namespace.
- push_children(ini_pdv, *children_pdv)[source]
push specified children projects to the origin remote.
- push_project(ini_pdv)[source]
push current/specified branch of project/package version-tagged to the remote host domain.
- Parameters:
ini_pdv¶ (
ProjectDevVars) – project dev vars.
- release_children(ini_pdv, *children_pdv)[source]
release the latest versions of the specified parent/root children projects to the origin remote.
- release_project(ini_pdv, version_tag)[source]
update local main branch from origin, optionally release (to PyPI if pip_name is set) and mirror to GitHub.
- Parameters:
ini_pdv¶ (
ProjectDevVars) – project dev vars.version_tag¶ (
str) – push version tag in the formatv<version-number>to release orLATESTto use the version tag of the latest git repository version.
- request_children_merge(ini_pdv, *children_pdv)[source]
request specified children merge of a parent/namespace on the upstream/forked remote.
- request_merge(ini_pdv)[source]
request merge of the origin=fork repository into the main branch at the upstream/forked remote.
- search_repos(ini_pdv, fragment='')[source]
search remote repositories via a text fragment in its project name/description.
- show_children_status(ini_pdv, *children_pdv)[source]
display the local and remote status of parent/root children repos.
- web_app_version(connection)[source]
determine the version of a deployed django project package.
- Parameters:
connection¶ (
PythonanywhereApi) – established connection to the *.pythonanywhere.com server.- Return type:
- Returns:
version string of the package deployed to the web host/server or empty string if package version file or version-in-file not found.
- class PythonanywhereCom[source]
Bases:
RemoteHostremote actions on remote web host pythonanywhere.com (to be specified by –web_domain option).
- connection: PythonanywhereApi
requests http connection
- connect(ini_pdv)[source]
connect to www. and eu.pythonanywhere.com web host.
- Parameters:
ini_pdv¶ (
ProjectDevVars) – parent/root project dev vars.- Return type:
- Returns:
boolean True on successful authentication else False.
- deploy_flags = {'ALL': False, 'CLEANUP': False, 'LEAN': False, 'MASKS': []}
optional flag names and default values for the actions
check_deploy()anddeploy_project()
- deploy_differences(ini_pdv, action, version_tag, **optional_flags)[source]
determine differences between the specified repository and web host/server (deployable and deletable files).
- Parameters:
ini_pdv¶ (
ProjectDevVars) – project dev vars.action¶ (
str) – pass ‘check’ to only check the differences between the specified repository and the web server/host, or ‘deploy’ to prepare the deployment of these differences.version_tag¶ (
str) – project package version to deploy. passLATESTto use the version tag of the latest repository version (PyPI release), orWORKTREEto deploy from the actual local project package version (including unstaged/untracked files).optional_flags¶ – optional command line arguments, documented in detail in the declaration of the action method parameter
check_deploy.optional_flags.
- Return type:
- Returns:
tuple of 2 strings and 2 sets. the first string contains a description of the project and the server to check/deploy-to, and the second the path to the project root folder. the two sets containing project file paths, relative to the local/temporary project root folder, the first one with the deployable files, and the 2nd one with the removable files.
- check_deploy(ini_pdv, version_tag, **optional_flags)[source]
check all project package files at the app/web server against the specified package version.
- Parameters:
ini_pdv¶ (
ProjectDevVars) – project dev vars.version_tag¶ (
str) – version tag in the formatv<version-number>to check orLATESTto check against the latest repository version orWORKTREEto check directly against the local work tree (with the locally added, unstaged and changed files).optional_flags¶ –
additional/optionally supported command line arguments:
ALLis including all deployable package files, instead of only the new, changed or deleted files in the specified repository.CLEANUPis checking for deletable files on the web server/host, e.g., after they got removed from the specified repository or work tree.LEANis reducing the deployable files sets to the minimum (using e.g., the functionskip_files_lean_web()), like e.g., the gettext.pofiles, themedia_iniroot folder and thestaticsubfolder with the initial static files of the web project.MASKSspecifies a list of file paths masks/pattern to be included in the repository files to check/deploy. to include e.g., the files of the static root folder specify this argument asMASKS="['static/**/*']". single files can be included too, by adding their possible file names to the list - only the found ones will be included. for example, to include the django database, you could add some possible DB file names to the list like in"MASKS=['static/**/*', 'db.sqlite', 'project.db']"
- deploy_project(ini_pdv, version_tag, **optional_flags)[source]
deploy code files of a django/app project version to the web-/app-server.
- Parameters:
ini_pdv¶ (
ProjectDevVars) – project dev vars.version_tag¶ (
str) – version tag in the formatv<version-number>to deploy orLATESTto use the tag of the latest repository version orWORKTREEto deploy directly from the local work tree (including locally added, unstaged and changed files).optional_flags¶ – optional command line arguments, documented in the
check_deploy()action.
- add_children_file(ini_pdv, file_name, rel_path, *children_pdv)[source]
add any file to the project working trees of parent/root and children/portions.
- Parameters:
ini_pdv¶ (
ProjectDevVars) – parent/root project dev vars.file_name¶ (
str) – source (template) file name (optional with a path).rel_path¶ (
str) – destination path relative to the project root.children_pdv¶ (
ProjectDevVars) – project dev vars of the children to process.
- Return type:
- Returns:
boolean True if the file got added to the parent/root and to all children, else False.
- add_file(ini_pdv, file_name, rel_path='.')[source]
add any file into the project working tree.
- Parameters:
ini_pdv¶ (
ProjectDevVars) – project dev vars.file_name¶ (
str) – file name to add (either with an absolut path, or relative to the project root). if the source is a template file, then the path prefixes of the base file name will be processed (with the project dev vars as template vars).rel_path¶ (
str) – optional relative folder/dir path in the destination project working tree, defaults to the project root folder.
- Return type:
- Returns:
boolean True if the file got added to the specified project, else False.
- build_gui_app(ini_pdv, **build_flags)[source]
build gui app with buildozer, add LIBS to make a clean/full build and EMBED to include APK to share.
- check_children_integrity(parent_pdv, *children_pdv)[source]
run integrity checks for the specified children of a parent or portions of a namespace.
- check_integrity(ini_pdv)[source]
integrity check of files/folders completeness, managed/template files update-state, and CI tests.
- clone_children(parent_or_root_pdv, *owner_name_versions)[source]
clone specified namespace-portion/parent-child repositories to the local machine.
Hint
the supported command line options are documented in the
clone_project()action.- Parameters:
parent_or_root_pdv¶ (
ProjectDevVars) – parent/namespace-root project to clone from.the projects/packages/portions to be cloned, identified by their repository owner user|group, the project/portion name and an optional version:
group-name/project_name1==v3.6.9 project_name2 ...
running in namespace root/sister project allows to only specify the portion names:
portion_name1 portion_name2 ...
the user|group name is only obligatory when this action got started from a parent folder (else it defaults to owner of the namespace root|sister project).
- Return type:
- Returns:
cloned children project paths list (for :func_`clone_children` and unit testing).
- clone_project(ini_pdv, owner_name_version)[source]
clone remote repo to the local machine.
the origin host domain can be specified with the –repo_domain option. the owner user|group name can alternatively be specified via the –repo_group option. if the –branch option is specified, then only this branch/tag will be cloned (quicker!) and directly checked-out. extra checks on the correct portion/project name can be activated by specifying the –namespace_name option.
- Parameters:
ini_pdv¶ (
ProjectDevVars) – project vars, for path, owner and namespace defaults, either manually prepared for the project to clone or use the local parent or namespace root/sister project.owner_name_version¶ (
str) – name of the project to clone, optionally prefixed with the owner name (and a slash) and suffixed with (thePROJECT_VERSION_SEPseperator and) a version number.
- Return type:
- Returns:
project path of the cloned project or an empty string if an error occurred. needed/used by
clone_children()and unit tests.
- commit_children(ini_pdv, *children_pdv)[source]
commit changes to children of a namespace/parent using the individually prepared commit message files.
- commit_project(ini_pdv)[source]
commit changes of a single project to the local repo using the prepared commit message file.
- delete_children_file(ini_pdv, file_name, *children_pdv)[source]
delete a file or an empty folder from parent/root and children/portions project roots.
- Parameters:
ini_pdv¶ (
ProjectDevVars) – parent/root project dev vars.file_name¶ (
str) – file/folder name to delete (optional with a path, relative to the project root).children_pdv¶ (
ProjectDevVars) – tuple of children project dev vars.
- Return type:
- Returns:
boolean True if the file got found and deleted from the parent and all the children projects, else False.
- delete_file(ini_pdv, file_or_dir)[source]
delete a file or an empty folder from the project.
- Parameters:
ini_pdv¶ (
ProjectDevVars) – project dev vars.file_or_dir¶ (
str) – file/folder name to delete (optional with a path, relative to the project root).
- Return type:
- Returns:
boolean True if the file got found and deleted from the specified project, else False.
- install_children_editable(ini_pdv, *children_pdv)[source]
install parent children or namespace portions as editable on the local machine.
- install_editable(ini_pdv)[source]
install the project as editable from the source/project root folder.
- new_children(ini_pdv, *children_pdv)[source]
initialize or renew parent folder children or namespace portions.
- Return type:
- new_namespace_root(ini_pdv)[source]
create or complete/renew a namespace root package.
- Return type:
- prepare_children_commit(ini_pdv, title, *children_pdv)[source]
run code checks and prepare/overwrite the commit message file for a bulk-commit of children projects.
- Parameters:
ini_pdv¶ (
ProjectDevVars) – parent/root project dev vars.children_pdv¶ (
ProjectDevVars) – project dev var args tuple of the children to process.
- prepare_commit(ini_pdv, title='')[source]
run code checks and prepare/overwrite the commit message file for the commit of a single project/package.
- Parameters:
ini_pdv¶ (
ProjectDevVars) – project dev vars.title¶ (
str) – optional commit message title (with the f-string placeholder {project_version}).
- refresh_children_managed(ini_pdv, *children_pdv)[source]
refresh frozen requirements and managed files from templates in all the children projects.
- refresh_project(ini_pdv)[source]
refresh/renew all the *requirements_frozen.txt files and all the managed files of the specified project.
- rename_children_file(ini_pdv, old_file_name, new_file_name, *children_pdv)[source]
rename a file or folder in parent/root and children/portions projects.
- Parameters:
ini_pdv¶ (
ProjectDevVars) – parent/root project dev vars.old_file_name¶ (
str) – file/folder name to rename (optional with a path, relative to the project root).new_file_name¶ (
str) – new name of file/folder (optional with a path, relative to the project root).children_pdv¶ (
ProjectDevVars) – project dev vars tuple of the children to process.
- Return type:
- Returns:
boolean True if the file got renamed in the parent and all the children projects, else False.
- rename_file(ini_pdv, old_file_name, new_file_name)[source]
rename a file or folder in the project tree.
- Parameters:
- Return type:
- Returns:
boolean True if the file/folder got renamed, else False.
- renew_children(ini_pdv, *children_pdv)[source]
complete/renew/update the local children projects of the specified parent/namespace-root.
- run_children_command(ini_pdv, command, *children_pdv)[source]
run console command for the specified portions/children of a namespace/parent.
- Parameters:
ini_pdv¶ (
ProjectDevVars) – parent/root project dev vars.command¶ (
str) – console command string (including all command arguments).children_pdv¶ (
ProjectDevVars) – tuple of children project dev vars.
- show_actions(ini_pdv)[source]
get available/registered/implemented actions info of the specified/current project and remote.
- show_children_versions(ini_pdv, *children_pdv)[source]
show package versions (local, remote and on pypi) for the specified children of a namespace/parent.
- show_versions(ini_pdv)[source]
display package versions of worktree, remote repo(s), latest PyPI release and default app/web host.
- update_mirror(ini_pdv, mirror_remote)[source]
create or update a mirror of the actual repo onto the specified remote/host.
- Parameters:
ini_pdv¶ (
ProjectDevVars) – project dev vars of the project to create/update a mirror/replication for.mirror_remote¶ (
str) – mirror remote name or server/host url (optionally with authentication) to push to.
Note
there are three more pushable (but currently not implemented) git ref namespaces: pull, pipelines and lfs. other git ref namespaces are stash and remotes (remotes cannot be pushed - therefore the git push option –mirror cannot be used to create&update a mirror at GitHub/GitLab).
- upgrade_requirements(ini_pdv, **optional_flags)[source]
upgrade project requirements|dependencies, optionally as editable package.
- Parameters:
ini_pdv¶ (
ProjectDevVars) – project dev vars of the project to create/update a mirror/replication for.optional_flags¶ –
additional/optional command line arguments:
EDITABLE: requirements available as sister packager (under the same project parent folder) will be installed/upgraded as editable (via the -e option of pip).MASKS: list of package name masks/pattern strings to restrict the upgraded packages. if not specified then all packages required by the project will be upgraded.