mirror of
https://github.com/Dev-Wiki/git-repo.git
synced 2025-08-20 19:59:13 +08:00
Compare commits
No commits in common. "master" and "v2.4.1" have entirely different histories.
5
.github/workflows/test-ci.yml
vendored
5
.github/workflows/test-ci.yml
vendored
@ -14,7 +14,10 @@ jobs:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
os: [ubuntu-latest, macos-latest, windows-latest]
|
||||
python-version: [3.6, 3.7, 3.8]
|
||||
python-version: [2.7, 3.6, 3.7, 3.8]
|
||||
exclude:
|
||||
- os: windows-latest
|
||||
python-version: 2.7
|
||||
runs-on: ${{ matrix.os }}
|
||||
|
||||
steps:
|
||||
|
||||
17
README.md
17
README.md
@ -7,7 +7,6 @@ easier to work with Git. The repo command is an executable Python script
|
||||
that you can put anywhere in your path.
|
||||
|
||||
* Homepage: <https://gerrit.googlesource.com/git-repo/>
|
||||
* Mailing list: [repo-discuss on Google Groups][repo-discuss]
|
||||
* Bug reports: <https://bugs.chromium.org/p/gerrit/issues/list?q=component:repo>
|
||||
* Source: <https://gerrit.googlesource.com/git-repo/>
|
||||
* Overview: <https://source.android.com/source/developing.html>
|
||||
@ -19,17 +18,6 @@ that you can put anywhere in your path.
|
||||
* GitHub mirror: <https://github.com/GerritCodeReview/git-repo>
|
||||
* Postsubmit tests: <https://github.com/GerritCodeReview/git-repo/actions>
|
||||
|
||||
## Contact
|
||||
|
||||
Please use the [repo-discuss] mailing list or [issue tracker] for questions.
|
||||
|
||||
You can [file a new bug report][new-bug] under the "repo" component.
|
||||
|
||||
Please do not e-mail individual developers for support.
|
||||
They do not have the bandwidth for it, and often times questions have already
|
||||
been asked on [repo-discuss] or bugs posted to the [issue tracker].
|
||||
So please search those sites first.
|
||||
|
||||
## Install
|
||||
|
||||
Many distros include repo, so you might be able to install from there.
|
||||
@ -48,8 +36,3 @@ $ PATH="${HOME}/.bin:${PATH}"
|
||||
$ curl https://storage.googleapis.com/git-repo-downloads/repo > ~/.bin/repo
|
||||
$ chmod a+rx ~/.bin/repo
|
||||
```
|
||||
|
||||
|
||||
[new-bug]: https://bugs.chromium.org/p/gerrit/issues/entry?template=Repo+tool+issue
|
||||
[issue tracker]: https://bugs.chromium.org/p/gerrit/issues/list?q=component:repo
|
||||
[repo-discuss]: https://groups.google.com/forum/#!forum/repo-discuss
|
||||
|
||||
@ -34,7 +34,7 @@ For example, if you want to change the manifest branch, you can simply run
|
||||
|
||||
It tracks the git repository at `REPO_URL` using the `REPO_REV` branch.
|
||||
Those are specified at `repo init` time using the `--repo-url=<REPO_URL>`
|
||||
and `--repo-rev=<REPO_REV>` options.
|
||||
and `--repo-branch=<REPO_REV>` options.
|
||||
|
||||
Any changes made to this directory will usually be automatically discarded
|
||||
by repo itself when it checks for updates. If you want to update to the
|
||||
@ -134,7 +134,6 @@ User controlled settings are initialized when running `repo init`.
|
||||
|-------------------|---------------------------|-------------|
|
||||
| manifest.groups | `--groups` & `--platform` | The manifest groups to sync |
|
||||
| repo.archive | `--archive` | Use `git archive` for checkouts |
|
||||
| repo.clonebundle | `--clone-bundle` | Whether the initial sync used clone.bundle explicitly |
|
||||
| repo.clonefilter | `--clone-filter` | Filter setting when using [partial git clones] |
|
||||
| repo.depth | `--depth` | Create shallow checkouts when cloning |
|
||||
| repo.dissociate | `--dissociate` | Dissociate from any reference/mirrors after initial clone |
|
||||
@ -194,9 +193,7 @@ The `[branch]` settings are updated by `repo start` and `git branch`.
|
||||
| review.\<url\>.autocopy | upload | Automatically add to `--cc=<value>` |
|
||||
| review.\<url\>.autoreviewer | upload | Automatically add to `--reviewers=<value>` |
|
||||
| review.\<url\>.autoupload | upload | Automatically answer "yes" or "no" to all prompts |
|
||||
| review.\<url\>.uploadhashtags | upload | Automatically add to `--hashtag=<value>` |
|
||||
| review.\<url\>.uploadlabels | upload | Automatically add to `--label=<value>` |
|
||||
| review.\<url\>.uploadnotify | upload | [Notify setting][upload-notify] to use |
|
||||
| review.\<url\>.uploadhashtags | upload | Automatically add to `--hashtags=<value>` |
|
||||
| review.\<url\>.uploadtopic | upload | Default [topic] to use |
|
||||
| review.\<url\>.username | upload | Override username with `ssh://` review URIs |
|
||||
| remote.\<remote\>.fetch | sync | Set of refs to fetch |
|
||||
@ -229,4 +226,3 @@ Repo will create & maintain a few files in the user's home directory.
|
||||
[manifest-format.md]: ./manifest-format.md
|
||||
[local manifests]: ./manifest-format.md#Local-Manifests
|
||||
[topic]: https://gerrit-review.googlesource.com/Documentation/intro-user.html#topics
|
||||
[upload-notify]: https://gerrit-review.googlesource.com/Documentation/user-upload.html#notify
|
||||
|
||||
@ -396,4 +396,10 @@ these extra projects.
|
||||
Manifest files stored in `$TOP_DIR/.repo/local_manifests/*.xml` will
|
||||
be loaded in alphabetical order.
|
||||
|
||||
The legacy `$TOP_DIR/.repo/local_manifest.xml` path is no longer supported.
|
||||
Additional remotes and projects may also be added through a local
|
||||
manifest, stored in `$TOP_DIR/.repo/local_manifest.xml`. This method
|
||||
is deprecated in favor of using multiple manifest files as mentioned
|
||||
above.
|
||||
|
||||
If `$TOP_DIR/.repo/local_manifest.xml` exists, it will be loaded before
|
||||
any manifest files stored in `$TOP_DIR/.repo/local_manifests/*.xml`.
|
||||
|
||||
@ -5,37 +5,6 @@ related topics and flows.
|
||||
|
||||
[TOC]
|
||||
|
||||
## Schedule
|
||||
|
||||
There is no specific schedule for when releases are made.
|
||||
Usually it's more along the lines of "enough minor changes have been merged",
|
||||
or "there's a known issue the maintainers know should get fixed".
|
||||
If you find a fix has been merged for an issue important to you, but hasn't been
|
||||
released after a week or so, feel free to [contact] us to request a new release.
|
||||
|
||||
### Release Freezes {#freeze}
|
||||
|
||||
We try to observe a regular schedule for when **not** to release.
|
||||
If something goes wrong, staff need to be active in order to respond quickly &
|
||||
effectively.
|
||||
We also don't want to disrupt non-Google organizations if possible.
|
||||
|
||||
We generally follow the rules:
|
||||
|
||||
* Release during Mon - Thu, 9:00 - 14:00 [US PT]
|
||||
* Avoid holidays
|
||||
* All regular [US holidays]
|
||||
* Large international ones if possible
|
||||
* All the various [New Years]
|
||||
* Jan 1 in Gregorian calendar is the most obvious
|
||||
* Check for large Lunar New Years too
|
||||
* Follow the normal [Google production freeze schedule]
|
||||
|
||||
[US holidays]: https://en.wikipedia.org/wiki/Federal_holidays_in_the_United_States
|
||||
[US PT]: https://en.wikipedia.org/wiki/Pacific_Time_Zone
|
||||
[New Years]: https://en.wikipedia.org/wiki/New_Year
|
||||
[Google production freeze schedule]: http://goto.google.com/prod-freeze
|
||||
|
||||
## Launcher script
|
||||
|
||||
The main repo script serves as a standalone program and is often referred to as
|
||||
@ -80,11 +49,11 @@ control how repo finds updates:
|
||||
|
||||
* `--repo-url`: This tells repo where to clone the full repo project itself.
|
||||
It defaults to the official project (`REPO_URL` in the launcher script).
|
||||
* `--repo-rev`: This tells repo which branch to use for the full project.
|
||||
* `--repo-branch`: This tells repo which branch to use for the full project.
|
||||
It defaults to the `stable` branch (`REPO_REV` in the launcher script).
|
||||
|
||||
Whenever `repo sync` is run, repo will check to see if an update is available.
|
||||
It fetches the latest repo-rev from the repo-url.
|
||||
It fetches the latest repo-branch from the repo-url.
|
||||
Then it verifies that the latest commit in the branch has a valid signed tag
|
||||
using `git tag -v` (which uses gpg).
|
||||
If the tag is valid, then repo will update its internal checkout to it.
|
||||
@ -122,7 +91,7 @@ When you want to create a new release, you'll need to select a good version and
|
||||
create a signed tag using a key registered in repo itself.
|
||||
Typically we just tag the latest version of the `master` branch.
|
||||
The tag could be pushed now, but it won't be used by clients normally (since the
|
||||
default `repo-rev` setting is `stable`).
|
||||
default `repo-branch` setting is `stable`).
|
||||
This would allow some early testing on systems who explicitly select `master`.
|
||||
|
||||
### Creating a signed tag
|
||||
@ -273,7 +242,6 @@ Things in italics are things we used to care about but probably don't anymore.
|
||||
| Apr 2020 | **Apr 2030** | | | **20.04 Focal** | 2.25.0 | 2.7.17 3.7.5 |
|
||||
|
||||
|
||||
[contact]: ../README.md#contact
|
||||
[rel-d]: https://en.wikipedia.org/wiki/Debian_version_history
|
||||
[rel-g]: https://en.wikipedia.org/wiki/Git#Releases
|
||||
[rel-p]: https://en.wikipedia.org/wiki/History_of_Python#Table_of_versions
|
||||
|
||||
@ -16,7 +16,6 @@
|
||||
|
||||
from __future__ import print_function
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
import subprocess
|
||||
import tempfile
|
||||
@ -48,35 +47,6 @@ LAST_CWD = None
|
||||
_ssh_proxy_path = None
|
||||
_ssh_sock_path = None
|
||||
_ssh_clients = []
|
||||
_ssh_version = None
|
||||
|
||||
|
||||
def _run_ssh_version():
|
||||
"""run ssh -V to display the version number"""
|
||||
return subprocess.check_output(['ssh', '-V'], stderr=subprocess.STDOUT).decode()
|
||||
|
||||
|
||||
def _parse_ssh_version(ver_str=None):
|
||||
"""parse a ssh version string into a tuple"""
|
||||
if ver_str is None:
|
||||
ver_str = _run_ssh_version()
|
||||
m = re.match(r'^OpenSSH_([0-9.]+)(p[0-9]+)?\s', ver_str)
|
||||
if m:
|
||||
return tuple(int(x) for x in m.group(1).split('.'))
|
||||
else:
|
||||
return ()
|
||||
|
||||
|
||||
def ssh_version():
|
||||
"""return ssh version as a tuple"""
|
||||
global _ssh_version
|
||||
if _ssh_version is None:
|
||||
try:
|
||||
_ssh_version = _parse_ssh_version()
|
||||
except subprocess.CalledProcessError:
|
||||
print('fatal: unable to detect ssh version', file=sys.stderr)
|
||||
sys.exit(1)
|
||||
return _ssh_version
|
||||
|
||||
|
||||
def ssh_sock(create=True):
|
||||
@ -87,13 +57,9 @@ def ssh_sock(create=True):
|
||||
tmp_dir = '/tmp'
|
||||
if not os.path.exists(tmp_dir):
|
||||
tmp_dir = tempfile.gettempdir()
|
||||
if ssh_version() < (6, 7):
|
||||
tokens = '%r@%h:%p'
|
||||
else:
|
||||
tokens = '%C' # hash of %l%h%p%r
|
||||
_ssh_sock_path = os.path.join(
|
||||
tempfile.mkdtemp('', 'ssh-', tmp_dir),
|
||||
'master-' + tokens)
|
||||
'master-%r@%h:%p')
|
||||
return _ssh_sock_path
|
||||
|
||||
|
||||
|
||||
@ -362,7 +362,7 @@ class GitConfig(object):
|
||||
return c
|
||||
|
||||
def _do(self, *args):
|
||||
command = ['config', '--file', self.file, '--includes']
|
||||
command = ['config', '--file', self.file]
|
||||
command.extend(args)
|
||||
|
||||
p = GitCommand(None,
|
||||
|
||||
@ -23,8 +23,6 @@ R_CHANGES = 'refs/changes/'
|
||||
R_HEADS = 'refs/heads/'
|
||||
R_TAGS = 'refs/tags/'
|
||||
R_PUB = 'refs/published/'
|
||||
R_WORKTREE = 'refs/worktree/'
|
||||
R_WORKTREE_M = R_WORKTREE + 'm/'
|
||||
R_M = 'refs/remotes/m/'
|
||||
|
||||
|
||||
|
||||
431
hooks.py
431
hooks.py
@ -1,431 +0,0 @@
|
||||
# -*- coding:utf-8 -*-
|
||||
#
|
||||
# Copyright (C) 2008 The Android Open Source Project
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
import traceback
|
||||
|
||||
from error import HookError
|
||||
from git_refs import HEAD
|
||||
|
||||
from pyversion import is_python3
|
||||
if is_python3():
|
||||
import urllib.parse
|
||||
else:
|
||||
import imp
|
||||
import urlparse
|
||||
urllib = imp.new_module('urllib')
|
||||
urllib.parse = urlparse
|
||||
input = raw_input # noqa: F821
|
||||
|
||||
class RepoHook(object):
|
||||
"""A RepoHook contains information about a script to run as a hook.
|
||||
|
||||
Hooks are used to run a python script before running an upload (for instance,
|
||||
to run presubmit checks). Eventually, we may have hooks for other actions.
|
||||
|
||||
This shouldn't be confused with files in the 'repo/hooks' directory. Those
|
||||
files are copied into each '.git/hooks' folder for each project. Repo-level
|
||||
hooks are associated instead with repo actions.
|
||||
|
||||
Hooks are always python. When a hook is run, we will load the hook into the
|
||||
interpreter and execute its main() function.
|
||||
"""
|
||||
|
||||
def __init__(self,
|
||||
hook_type,
|
||||
hooks_project,
|
||||
topdir,
|
||||
manifest_url,
|
||||
abort_if_user_denies=False):
|
||||
"""RepoHook constructor.
|
||||
|
||||
Params:
|
||||
hook_type: A string representing the type of hook. This is also used
|
||||
to figure out the name of the file containing the hook. For
|
||||
example: 'pre-upload'.
|
||||
hooks_project: The project containing the repo hooks. If you have a
|
||||
manifest, this is manifest.repo_hooks_project. OK if this is None,
|
||||
which will make the hook a no-op.
|
||||
topdir: Repo's top directory (the one containing the .repo directory).
|
||||
Scripts will run with CWD as this directory. If you have a manifest,
|
||||
this is manifest.topdir
|
||||
manifest_url: The URL to the manifest git repo.
|
||||
abort_if_user_denies: If True, we'll throw a HookError() if the user
|
||||
doesn't allow us to run the hook.
|
||||
"""
|
||||
self._hook_type = hook_type
|
||||
self._hooks_project = hooks_project
|
||||
self._manifest_url = manifest_url
|
||||
self._topdir = topdir
|
||||
self._abort_if_user_denies = abort_if_user_denies
|
||||
|
||||
# Store the full path to the script for convenience.
|
||||
if self._hooks_project:
|
||||
self._script_fullpath = os.path.join(self._hooks_project.worktree,
|
||||
self._hook_type + '.py')
|
||||
else:
|
||||
self._script_fullpath = None
|
||||
|
||||
def _GetHash(self):
|
||||
"""Return a hash of the contents of the hooks directory.
|
||||
|
||||
We'll just use git to do this. This hash has the property that if anything
|
||||
changes in the directory we will return a different has.
|
||||
|
||||
SECURITY CONSIDERATION:
|
||||
This hash only represents the contents of files in the hook directory, not
|
||||
any other files imported or called by hooks. Changes to imported files
|
||||
can change the script behavior without affecting the hash.
|
||||
|
||||
Returns:
|
||||
A string representing the hash. This will always be ASCII so that it can
|
||||
be printed to the user easily.
|
||||
"""
|
||||
assert self._hooks_project, "Must have hooks to calculate their hash."
|
||||
|
||||
# We will use the work_git object rather than just calling GetRevisionId().
|
||||
# That gives us a hash of the latest checked in version of the files that
|
||||
# the user will actually be executing. Specifically, GetRevisionId()
|
||||
# doesn't appear to change even if a user checks out a different version
|
||||
# of the hooks repo (via git checkout) nor if a user commits their own revs.
|
||||
#
|
||||
# NOTE: Local (non-committed) changes will not be factored into this hash.
|
||||
# I think this is OK, since we're really only worried about warning the user
|
||||
# about upstream changes.
|
||||
return self._hooks_project.work_git.rev_parse('HEAD')
|
||||
|
||||
def _GetMustVerb(self):
|
||||
"""Return 'must' if the hook is required; 'should' if not."""
|
||||
if self._abort_if_user_denies:
|
||||
return 'must'
|
||||
else:
|
||||
return 'should'
|
||||
|
||||
def _CheckForHookApproval(self):
|
||||
"""Check to see whether this hook has been approved.
|
||||
|
||||
We'll accept approval of manifest URLs if they're using secure transports.
|
||||
This way the user can say they trust the manifest hoster. For insecure
|
||||
hosts, we fall back to checking the hash of the hooks repo.
|
||||
|
||||
Note that we ask permission for each individual hook even though we use
|
||||
the hash of all hooks when detecting changes. We'd like the user to be
|
||||
able to approve / deny each hook individually. We only use the hash of all
|
||||
hooks because there is no other easy way to detect changes to local imports.
|
||||
|
||||
Returns:
|
||||
True if this hook is approved to run; False otherwise.
|
||||
|
||||
Raises:
|
||||
HookError: Raised if the user doesn't approve and abort_if_user_denies
|
||||
was passed to the consturctor.
|
||||
"""
|
||||
if self._ManifestUrlHasSecureScheme():
|
||||
return self._CheckForHookApprovalManifest()
|
||||
else:
|
||||
return self._CheckForHookApprovalHash()
|
||||
|
||||
def _CheckForHookApprovalHelper(self, subkey, new_val, main_prompt,
|
||||
changed_prompt):
|
||||
"""Check for approval for a particular attribute and hook.
|
||||
|
||||
Args:
|
||||
subkey: The git config key under [repo.hooks.<hook_type>] to store the
|
||||
last approved string.
|
||||
new_val: The new value to compare against the last approved one.
|
||||
main_prompt: Message to display to the user to ask for approval.
|
||||
changed_prompt: Message explaining why we're re-asking for approval.
|
||||
|
||||
Returns:
|
||||
True if this hook is approved to run; False otherwise.
|
||||
|
||||
Raises:
|
||||
HookError: Raised if the user doesn't approve and abort_if_user_denies
|
||||
was passed to the consturctor.
|
||||
"""
|
||||
hooks_config = self._hooks_project.config
|
||||
git_approval_key = 'repo.hooks.%s.%s' % (self._hook_type, subkey)
|
||||
|
||||
# Get the last value that the user approved for this hook; may be None.
|
||||
old_val = hooks_config.GetString(git_approval_key)
|
||||
|
||||
if old_val is not None:
|
||||
# User previously approved hook and asked not to be prompted again.
|
||||
if new_val == old_val:
|
||||
# Approval matched. We're done.
|
||||
return True
|
||||
else:
|
||||
# Give the user a reason why we're prompting, since they last told
|
||||
# us to "never ask again".
|
||||
prompt = 'WARNING: %s\n\n' % (changed_prompt,)
|
||||
else:
|
||||
prompt = ''
|
||||
|
||||
# Prompt the user if we're not on a tty; on a tty we'll assume "no".
|
||||
if sys.stdout.isatty():
|
||||
prompt += main_prompt + ' (yes/always/NO)? '
|
||||
response = input(prompt).lower()
|
||||
print()
|
||||
|
||||
# User is doing a one-time approval.
|
||||
if response in ('y', 'yes'):
|
||||
return True
|
||||
elif response == 'always':
|
||||
hooks_config.SetString(git_approval_key, new_val)
|
||||
return True
|
||||
|
||||
# For anything else, we'll assume no approval.
|
||||
if self._abort_if_user_denies:
|
||||
raise HookError('You must allow the %s hook or use --no-verify.' %
|
||||
self._hook_type)
|
||||
|
||||
return False
|
||||
|
||||
def _ManifestUrlHasSecureScheme(self):
|
||||
"""Check if the URI for the manifest is a secure transport."""
|
||||
secure_schemes = ('file', 'https', 'ssh', 'persistent-https', 'sso', 'rpc')
|
||||
parse_results = urllib.parse.urlparse(self._manifest_url)
|
||||
return parse_results.scheme in secure_schemes
|
||||
|
||||
def _CheckForHookApprovalManifest(self):
|
||||
"""Check whether the user has approved this manifest host.
|
||||
|
||||
Returns:
|
||||
True if this hook is approved to run; False otherwise.
|
||||
"""
|
||||
return self._CheckForHookApprovalHelper(
|
||||
'approvedmanifest',
|
||||
self._manifest_url,
|
||||
'Run hook scripts from %s' % (self._manifest_url,),
|
||||
'Manifest URL has changed since %s was allowed.' % (self._hook_type,))
|
||||
|
||||
def _CheckForHookApprovalHash(self):
|
||||
"""Check whether the user has approved the hooks repo.
|
||||
|
||||
Returns:
|
||||
True if this hook is approved to run; False otherwise.
|
||||
"""
|
||||
prompt = ('Repo %s run the script:\n'
|
||||
' %s\n'
|
||||
'\n'
|
||||
'Do you want to allow this script to run')
|
||||
return self._CheckForHookApprovalHelper(
|
||||
'approvedhash',
|
||||
self._GetHash(),
|
||||
prompt % (self._GetMustVerb(), self._script_fullpath),
|
||||
'Scripts have changed since %s was allowed.' % (self._hook_type,))
|
||||
|
||||
@staticmethod
|
||||
def _ExtractInterpFromShebang(data):
|
||||
"""Extract the interpreter used in the shebang.
|
||||
|
||||
Try to locate the interpreter the script is using (ignoring `env`).
|
||||
|
||||
Args:
|
||||
data: The file content of the script.
|
||||
|
||||
Returns:
|
||||
The basename of the main script interpreter, or None if a shebang is not
|
||||
used or could not be parsed out.
|
||||
"""
|
||||
firstline = data.splitlines()[:1]
|
||||
if not firstline:
|
||||
return None
|
||||
|
||||
# The format here can be tricky.
|
||||
shebang = firstline[0].strip()
|
||||
m = re.match(r'^#!\s*([^\s]+)(?:\s+([^\s]+))?', shebang)
|
||||
if not m:
|
||||
return None
|
||||
|
||||
# If the using `env`, find the target program.
|
||||
interp = m.group(1)
|
||||
if os.path.basename(interp) == 'env':
|
||||
interp = m.group(2)
|
||||
|
||||
return interp
|
||||
|
||||
def _ExecuteHookViaReexec(self, interp, context, **kwargs):
|
||||
"""Execute the hook script through |interp|.
|
||||
|
||||
Note: Support for this feature should be dropped ~Jun 2021.
|
||||
|
||||
Args:
|
||||
interp: The Python program to run.
|
||||
context: Basic Python context to execute the hook inside.
|
||||
kwargs: Arbitrary arguments to pass to the hook script.
|
||||
|
||||
Raises:
|
||||
HookError: When the hooks failed for any reason.
|
||||
"""
|
||||
# This logic needs to be kept in sync with _ExecuteHookViaImport below.
|
||||
script = """
|
||||
import json, os, sys
|
||||
path = '''%(path)s'''
|
||||
kwargs = json.loads('''%(kwargs)s''')
|
||||
context = json.loads('''%(context)s''')
|
||||
sys.path.insert(0, os.path.dirname(path))
|
||||
data = open(path).read()
|
||||
exec(compile(data, path, 'exec'), context)
|
||||
context['main'](**kwargs)
|
||||
""" % {
|
||||
'path': self._script_fullpath,
|
||||
'kwargs': json.dumps(kwargs),
|
||||
'context': json.dumps(context),
|
||||
}
|
||||
|
||||
# We pass the script via stdin to avoid OS argv limits. It also makes
|
||||
# unhandled exception tracebacks less verbose/confusing for users.
|
||||
cmd = [interp, '-c', 'import sys; exec(sys.stdin.read())']
|
||||
proc = subprocess.Popen(cmd, stdin=subprocess.PIPE)
|
||||
proc.communicate(input=script.encode('utf-8'))
|
||||
if proc.returncode:
|
||||
raise HookError('Failed to run %s hook.' % (self._hook_type,))
|
||||
|
||||
def _ExecuteHookViaImport(self, data, context, **kwargs):
|
||||
"""Execute the hook code in |data| directly.
|
||||
|
||||
Args:
|
||||
data: The code of the hook to execute.
|
||||
context: Basic Python context to execute the hook inside.
|
||||
kwargs: Arbitrary arguments to pass to the hook script.
|
||||
|
||||
Raises:
|
||||
HookError: When the hooks failed for any reason.
|
||||
"""
|
||||
# Exec, storing global context in the context dict. We catch exceptions
|
||||
# and convert to a HookError w/ just the failing traceback.
|
||||
try:
|
||||
exec(compile(data, self._script_fullpath, 'exec'), context)
|
||||
except Exception:
|
||||
raise HookError('%s\nFailed to import %s hook; see traceback above.' %
|
||||
(traceback.format_exc(), self._hook_type))
|
||||
|
||||
# Running the script should have defined a main() function.
|
||||
if 'main' not in context:
|
||||
raise HookError('Missing main() in: "%s"' % self._script_fullpath)
|
||||
|
||||
# Call the main function in the hook. If the hook should cause the
|
||||
# build to fail, it will raise an Exception. We'll catch that convert
|
||||
# to a HookError w/ just the failing traceback.
|
||||
try:
|
||||
context['main'](**kwargs)
|
||||
except Exception:
|
||||
raise HookError('%s\nFailed to run main() for %s hook; see traceback '
|
||||
'above.' % (traceback.format_exc(), self._hook_type))
|
||||
|
||||
def _ExecuteHook(self, **kwargs):
|
||||
"""Actually execute the given hook.
|
||||
|
||||
This will run the hook's 'main' function in our python interpreter.
|
||||
|
||||
Args:
|
||||
kwargs: Keyword arguments to pass to the hook. These are often specific
|
||||
to the hook type. For instance, pre-upload hooks will contain
|
||||
a project_list.
|
||||
"""
|
||||
# Keep sys.path and CWD stashed away so that we can always restore them
|
||||
# upon function exit.
|
||||
orig_path = os.getcwd()
|
||||
orig_syspath = sys.path
|
||||
|
||||
try:
|
||||
# Always run hooks with CWD as topdir.
|
||||
os.chdir(self._topdir)
|
||||
|
||||
# Put the hook dir as the first item of sys.path so hooks can do
|
||||
# relative imports. We want to replace the repo dir as [0] so
|
||||
# hooks can't import repo files.
|
||||
sys.path = [os.path.dirname(self._script_fullpath)] + sys.path[1:]
|
||||
|
||||
# Initial global context for the hook to run within.
|
||||
context = {'__file__': self._script_fullpath}
|
||||
|
||||
# Add 'hook_should_take_kwargs' to the arguments to be passed to main.
|
||||
# We don't actually want hooks to define their main with this argument--
|
||||
# it's there to remind them that their hook should always take **kwargs.
|
||||
# For instance, a pre-upload hook should be defined like:
|
||||
# def main(project_list, **kwargs):
|
||||
#
|
||||
# This allows us to later expand the API without breaking old hooks.
|
||||
kwargs = kwargs.copy()
|
||||
kwargs['hook_should_take_kwargs'] = True
|
||||
|
||||
# See what version of python the hook has been written against.
|
||||
data = open(self._script_fullpath).read()
|
||||
interp = self._ExtractInterpFromShebang(data)
|
||||
reexec = False
|
||||
if interp:
|
||||
prog = os.path.basename(interp)
|
||||
if prog.startswith('python2') and sys.version_info.major != 2:
|
||||
reexec = True
|
||||
elif prog.startswith('python3') and sys.version_info.major == 2:
|
||||
reexec = True
|
||||
|
||||
# Attempt to execute the hooks through the requested version of Python.
|
||||
if reexec:
|
||||
try:
|
||||
self._ExecuteHookViaReexec(interp, context, **kwargs)
|
||||
except OSError as e:
|
||||
if e.errno == errno.ENOENT:
|
||||
# We couldn't find the interpreter, so fallback to importing.
|
||||
reexec = False
|
||||
else:
|
||||
raise
|
||||
|
||||
# Run the hook by importing directly.
|
||||
if not reexec:
|
||||
self._ExecuteHookViaImport(data, context, **kwargs)
|
||||
finally:
|
||||
# Restore sys.path and CWD.
|
||||
sys.path = orig_syspath
|
||||
os.chdir(orig_path)
|
||||
|
||||
def Run(self, user_allows_all_hooks, **kwargs):
|
||||
"""Run the hook.
|
||||
|
||||
If the hook doesn't exist (because there is no hooks project or because
|
||||
this particular hook is not enabled), this is a no-op.
|
||||
|
||||
Args:
|
||||
user_allows_all_hooks: If True, we will never prompt about running the
|
||||
hook--we'll just assume it's OK to run it.
|
||||
kwargs: Keyword arguments to pass to the hook. These are often specific
|
||||
to the hook type. For instance, pre-upload hooks will contain
|
||||
a project_list.
|
||||
|
||||
Raises:
|
||||
HookError: If there was a problem finding the hook or the user declined
|
||||
to run a required hook (from _CheckForHookApproval).
|
||||
"""
|
||||
# No-op if there is no hooks project or if hook is disabled.
|
||||
if ((not self._hooks_project) or (self._hook_type not in
|
||||
self._hooks_project.enabled_repo_hooks)):
|
||||
return
|
||||
|
||||
# Bail with a nice error if we can't find the hook.
|
||||
if not os.path.isfile(self._script_fullpath):
|
||||
raise HookError('Couldn\'t find repo hook: "%s"' % self._script_fullpath)
|
||||
|
||||
# Make sure the user is OK with running the hook.
|
||||
if (not user_allows_all_hooks) and (not self._CheckForHookApproval()):
|
||||
return
|
||||
|
||||
# Run the hook with the same version of python we're using.
|
||||
self._ExecuteHook(**kwargs)
|
||||
22
main.py
22
main.py
@ -135,6 +135,8 @@ class _Repo(object):
|
||||
def __init__(self, repodir):
|
||||
self.repodir = repodir
|
||||
self.commands = all_commands
|
||||
# add 'branch' as an alias for 'branches'
|
||||
all_commands['branch'] = all_commands['branches']
|
||||
|
||||
def _ParseArgs(self, argv):
|
||||
"""Parse the main `repo` command line options."""
|
||||
@ -204,7 +206,7 @@ class _Repo(object):
|
||||
SetDefaultColoring(gopts.color)
|
||||
|
||||
try:
|
||||
cmd = self.commands[name]()
|
||||
cmd = self.commands[name]
|
||||
except KeyError:
|
||||
print("repo: '%s' is not a repo command. See 'repo help'." % name,
|
||||
file=sys.stderr)
|
||||
@ -346,20 +348,12 @@ repo: error:
|
||||
sys.exit(1)
|
||||
|
||||
if exp > ver:
|
||||
print('\n... A new version of repo (%s) is available.' % (exp_str,),
|
||||
file=sys.stderr)
|
||||
if os.access(repo_path, os.W_OK):
|
||||
print("""\
|
||||
print("""
|
||||
... A new version of repo (%s) is available.
|
||||
... You should upgrade soon:
|
||||
|
||||
cp %s %s
|
||||
""" % (WrapperPath(), repo_path), file=sys.stderr)
|
||||
else:
|
||||
print("""\
|
||||
... New version is available at: %s
|
||||
... The launcher is run from: %s
|
||||
!!! The launcher is not writable. Please talk to your sysadmin or distro
|
||||
!!! to get an update installed.
|
||||
""" % (WrapperPath(), repo_path), file=sys.stderr)
|
||||
""" % (exp_str, WrapperPath(), repo_path), file=sys.stderr)
|
||||
|
||||
|
||||
def _CheckRepoDir(repo_dir):
|
||||
@ -614,7 +608,7 @@ def _Main(argv):
|
||||
argv = list(sys.argv)
|
||||
argv.extend(rce.extra_args)
|
||||
try:
|
||||
os.execv(sys.executable, [__file__] + argv)
|
||||
os.execv(__file__, argv)
|
||||
except OSError as e:
|
||||
print('fatal: cannot restart repo after upgrade', file=sys.stderr)
|
||||
print('fatal: %s' % e, file=sys.stderr)
|
||||
|
||||
172
manifest_xml.py
172
manifest_xml.py
@ -31,7 +31,7 @@ else:
|
||||
urllib.parse = urlparse
|
||||
|
||||
import gitc_utils
|
||||
from git_config import GitConfig, IsId
|
||||
from git_config import GitConfig
|
||||
from git_refs import R_HEADS, HEAD
|
||||
import platform_utils
|
||||
from project import RemoteSpec, Project, MetaProject
|
||||
@ -57,60 +57,6 @@ urllib.parse.uses_netloc.extend([
|
||||
'rpc'])
|
||||
|
||||
|
||||
def XmlBool(node, attr, default=None):
|
||||
"""Determine boolean value of |node|'s |attr|.
|
||||
|
||||
Invalid values will issue a non-fatal warning.
|
||||
|
||||
Args:
|
||||
node: XML node whose attributes we access.
|
||||
attr: The attribute to access.
|
||||
default: If the attribute is not set (value is empty), then use this.
|
||||
|
||||
Returns:
|
||||
True if the attribute is a valid string representing true.
|
||||
False if the attribute is a valid string representing false.
|
||||
|default| otherwise.
|
||||
"""
|
||||
value = node.getAttribute(attr)
|
||||
s = value.lower()
|
||||
if s == '':
|
||||
return default
|
||||
elif s in {'yes', 'true', '1'}:
|
||||
return True
|
||||
elif s in {'no', 'false', '0'}:
|
||||
return False
|
||||
else:
|
||||
print('warning: manifest: %s="%s": ignoring invalid XML boolean' %
|
||||
(attr, value), file=sys.stderr)
|
||||
return default
|
||||
|
||||
|
||||
def XmlInt(node, attr, default=None):
|
||||
"""Determine integer value of |node|'s |attr|.
|
||||
|
||||
Args:
|
||||
node: XML node whose attributes we access.
|
||||
attr: The attribute to access.
|
||||
default: If the attribute is not set (value is empty), then use this.
|
||||
|
||||
Returns:
|
||||
The number if the attribute is a valid number.
|
||||
|
||||
Raises:
|
||||
ManifestParseError: The number is invalid.
|
||||
"""
|
||||
value = node.getAttribute(attr)
|
||||
if not value:
|
||||
return default
|
||||
|
||||
try:
|
||||
return int(value)
|
||||
except ValueError:
|
||||
raise ManifestParseError('manifest: invalid %s="%s" integer' %
|
||||
(attr, value))
|
||||
|
||||
|
||||
class _Default(object):
|
||||
"""Project defaults within the manifest."""
|
||||
|
||||
@ -192,6 +138,7 @@ class XmlManifest(object):
|
||||
self.topdir = os.path.dirname(self.repodir)
|
||||
self.manifestFile = os.path.join(self.repodir, MANIFEST_FILE_NAME)
|
||||
self.globalConfig = GitConfig.ForUser()
|
||||
self.localManifestWarning = False
|
||||
self.isGitcClient = False
|
||||
self._load_local_manifests = True
|
||||
|
||||
@ -283,7 +230,7 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
|
||||
def _ParseGroups(self, groups):
|
||||
return [x for x in re.split(r'[,\s]+', groups) if x]
|
||||
|
||||
def Save(self, fd, peg_rev=False, peg_rev_upstream=True, peg_rev_dest_branch=True, groups=None):
|
||||
def Save(self, fd, peg_rev=False, peg_rev_upstream=True, groups=None):
|
||||
"""Write the current manifest out to the given file descriptor.
|
||||
"""
|
||||
mp = self.manifestProject
|
||||
@ -388,13 +335,6 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
|
||||
# Only save the origin if the origin is not a sha1, and the default
|
||||
# isn't our value
|
||||
e.setAttribute('upstream', p.revisionExpr)
|
||||
|
||||
if peg_rev_dest_branch:
|
||||
if p.dest_branch:
|
||||
e.setAttribute('dest-branch', p.dest_branch)
|
||||
elif value != p.revisionExpr:
|
||||
e.setAttribute('dest-branch', p.revisionExpr)
|
||||
|
||||
else:
|
||||
revision = self.remotes[p.remote.orig_name].revision or d.revisionExpr
|
||||
if not revision or revision != p.revisionExpr:
|
||||
@ -500,14 +440,6 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
|
||||
self._Load()
|
||||
return self._manifest_server
|
||||
|
||||
@property
|
||||
def CloneBundle(self):
|
||||
clone_bundle = self.manifestProject.config.GetBoolean('repo.clonebundle')
|
||||
if clone_bundle is None:
|
||||
return False if self.manifestProject.config.GetBoolean('repo.partialclone') else True
|
||||
else:
|
||||
return clone_bundle
|
||||
|
||||
@property
|
||||
def CloneFilter(self):
|
||||
if self.manifestProject.config.GetBoolean('repo.partialclone'):
|
||||
@ -554,12 +486,15 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
|
||||
self.manifestProject.worktree))
|
||||
|
||||
if self._load_local_manifests:
|
||||
if os.path.exists(os.path.join(self.repodir, LOCAL_MANIFEST_NAME)):
|
||||
print('error: %s is not supported; put local manifests in `%s`'
|
||||
'instead' % (LOCAL_MANIFEST_NAME,
|
||||
os.path.join(self.repodir, LOCAL_MANIFESTS_DIR_NAME)),
|
||||
file=sys.stderr)
|
||||
sys.exit(1)
|
||||
local = os.path.join(self.repodir, LOCAL_MANIFEST_NAME)
|
||||
if os.path.exists(local):
|
||||
if not self.localManifestWarning:
|
||||
self.localManifestWarning = True
|
||||
print('warning: %s is deprecated; put local manifests '
|
||||
'in `%s` instead' % (LOCAL_MANIFEST_NAME,
|
||||
os.path.join(self.repodir, LOCAL_MANIFESTS_DIR_NAME)),
|
||||
file=sys.stderr)
|
||||
nodes.append(self._ParseManifestXml(local, self.repodir))
|
||||
|
||||
local_dir = os.path.abspath(os.path.join(self.repodir,
|
||||
LOCAL_MANIFESTS_DIR_NAME))
|
||||
@ -705,10 +640,6 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
|
||||
p.groups.extend(groups)
|
||||
if revision:
|
||||
p.revisionExpr = revision
|
||||
if IsId(revision):
|
||||
p.revisionId = revision
|
||||
else:
|
||||
p.revisionId = None
|
||||
if remote:
|
||||
p.remote = remote.ToRemoteSpec(name)
|
||||
if node.nodeName == 'repo-hooks':
|
||||
@ -826,14 +757,29 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
|
||||
d.destBranchExpr = node.getAttribute('dest-branch') or None
|
||||
d.upstreamExpr = node.getAttribute('upstream') or None
|
||||
|
||||
d.sync_j = XmlInt(node, 'sync-j', 1)
|
||||
if d.sync_j <= 0:
|
||||
raise ManifestParseError('%s: sync-j must be greater than 0, not "%s"' %
|
||||
(self.manifestFile, d.sync_j))
|
||||
sync_j = node.getAttribute('sync-j')
|
||||
if sync_j == '' or sync_j is None:
|
||||
d.sync_j = 1
|
||||
else:
|
||||
d.sync_j = int(sync_j)
|
||||
|
||||
d.sync_c = XmlBool(node, 'sync-c', False)
|
||||
d.sync_s = XmlBool(node, 'sync-s', False)
|
||||
d.sync_tags = XmlBool(node, 'sync-tags', True)
|
||||
sync_c = node.getAttribute('sync-c')
|
||||
if not sync_c:
|
||||
d.sync_c = False
|
||||
else:
|
||||
d.sync_c = sync_c.lower() in ("yes", "true", "1")
|
||||
|
||||
sync_s = node.getAttribute('sync-s')
|
||||
if not sync_s:
|
||||
d.sync_s = False
|
||||
else:
|
||||
d.sync_s = sync_s.lower() in ("yes", "true", "1")
|
||||
|
||||
sync_tags = node.getAttribute('sync-tags')
|
||||
if not sync_tags:
|
||||
d.sync_tags = True
|
||||
else:
|
||||
d.sync_tags = sync_tags.lower() in ("yes", "true", "1")
|
||||
return d
|
||||
|
||||
def _ParseNotice(self, node):
|
||||
@ -910,15 +856,39 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
|
||||
raise ManifestParseError("project %s path cannot be absolute in %s" %
|
||||
(name, self.manifestFile))
|
||||
|
||||
rebase = XmlBool(node, 'rebase', True)
|
||||
sync_c = XmlBool(node, 'sync-c', False)
|
||||
sync_s = XmlBool(node, 'sync-s', self._default.sync_s)
|
||||
sync_tags = XmlBool(node, 'sync-tags', self._default.sync_tags)
|
||||
rebase = node.getAttribute('rebase')
|
||||
if not rebase:
|
||||
rebase = True
|
||||
else:
|
||||
rebase = rebase.lower() in ("yes", "true", "1")
|
||||
|
||||
clone_depth = XmlInt(node, 'clone-depth')
|
||||
if clone_depth is not None and clone_depth <= 0:
|
||||
raise ManifestParseError('%s: clone-depth must be greater than 0, not "%s"' %
|
||||
(self.manifestFile, clone_depth))
|
||||
sync_c = node.getAttribute('sync-c')
|
||||
if not sync_c:
|
||||
sync_c = False
|
||||
else:
|
||||
sync_c = sync_c.lower() in ("yes", "true", "1")
|
||||
|
||||
sync_s = node.getAttribute('sync-s')
|
||||
if not sync_s:
|
||||
sync_s = self._default.sync_s
|
||||
else:
|
||||
sync_s = sync_s.lower() in ("yes", "true", "1")
|
||||
|
||||
sync_tags = node.getAttribute('sync-tags')
|
||||
if not sync_tags:
|
||||
sync_tags = self._default.sync_tags
|
||||
else:
|
||||
sync_tags = sync_tags.lower() in ("yes", "true", "1")
|
||||
|
||||
clone_depth = node.getAttribute('clone-depth')
|
||||
if clone_depth:
|
||||
try:
|
||||
clone_depth = int(clone_depth)
|
||||
if clone_depth <= 0:
|
||||
raise ValueError()
|
||||
except ValueError:
|
||||
raise ManifestParseError('invalid clone-depth %s in %s' %
|
||||
(clone_depth, self.manifestFile))
|
||||
|
||||
dest_branch = node.getAttribute('dest-branch') or self._default.destBranchExpr
|
||||
|
||||
@ -941,7 +911,7 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
|
||||
groups.extend(set(default_groups).difference(groups))
|
||||
|
||||
if self.IsMirror and node.hasAttribute('force-path'):
|
||||
if XmlBool(node, 'force-path', False):
|
||||
if node.getAttribute('force-path').lower() in ("yes", "true", "1"):
|
||||
gitdir = os.path.join(self.topdir, '%s.git' % path)
|
||||
|
||||
project = Project(manifest=self,
|
||||
@ -978,10 +948,6 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
|
||||
return project
|
||||
|
||||
def GetProjectPaths(self, name, path):
|
||||
# The manifest entries might have trailing slashes. Normalize them to avoid
|
||||
# unexpected filesystem behavior since we do string concatenation below.
|
||||
path = path.rstrip('/')
|
||||
name = name.rstrip('/')
|
||||
use_git_worktrees = False
|
||||
relpath = path
|
||||
if self.IsMirror:
|
||||
@ -1014,10 +980,6 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
|
||||
return os.path.relpath(relpath, parent_relpath)
|
||||
|
||||
def GetSubprojectPaths(self, parent, name, path):
|
||||
# The manifest entries might have trailing slashes. Normalize them to avoid
|
||||
# unexpected filesystem behavior since we do string concatenation below.
|
||||
path = path.rstrip('/')
|
||||
name = name.rstrip('/')
|
||||
relpath = self._JoinRelpath(parent.relpath, path)
|
||||
gitdir = os.path.join(parent.gitdir, 'subprojects', '%s.git' % path)
|
||||
objdir = os.path.join(parent.gitdir, 'subproject-objects', '%s.git' % name)
|
||||
|
||||
@ -90,11 +90,6 @@ class _FileDescriptorStreamsNonBlocking(FileDescriptorStreams):
|
||||
""" Implementation of FileDescriptorStreams for platforms that support
|
||||
non blocking I/O.
|
||||
"""
|
||||
def __init__(self):
|
||||
super(_FileDescriptorStreamsNonBlocking, self).__init__()
|
||||
self._poll = select.poll()
|
||||
self._fd_to_stream = {}
|
||||
|
||||
class Stream(object):
|
||||
""" Encapsulates a file descriptor """
|
||||
|
||||
@ -119,18 +114,11 @@ class _FileDescriptorStreamsNonBlocking(FileDescriptorStreams):
|
||||
self.fd.close()
|
||||
|
||||
def _create_stream(self, fd, dest, std_name):
|
||||
stream = self.Stream(fd, dest, std_name)
|
||||
self._fd_to_stream[stream.fileno()] = stream
|
||||
self._poll.register(stream, select.POLLIN)
|
||||
return stream
|
||||
|
||||
def remove(self, stream):
|
||||
self._poll.unregister(stream)
|
||||
del self._fd_to_stream[stream.fileno()]
|
||||
super(_FileDescriptorStreamsNonBlocking, self).remove(stream)
|
||||
return self.Stream(fd, dest, std_name)
|
||||
|
||||
def select(self):
|
||||
return [self._fd_to_stream[fd] for fd, _ in self._poll.poll()]
|
||||
ready_streams, _, _ = select.select(self.streams, [], [])
|
||||
return ready_streams
|
||||
|
||||
|
||||
class _FileDescriptorStreamsThreads(FileDescriptorStreams):
|
||||
|
||||
568
project.py
568
project.py
@ -18,6 +18,7 @@ from __future__ import print_function
|
||||
import errno
|
||||
import filecmp
|
||||
import glob
|
||||
import json
|
||||
import os
|
||||
import random
|
||||
import re
|
||||
@ -28,19 +29,20 @@ import sys
|
||||
import tarfile
|
||||
import tempfile
|
||||
import time
|
||||
import traceback
|
||||
|
||||
from color import Coloring
|
||||
from git_command import GitCommand, git_require
|
||||
from git_config import GitConfig, IsId, GetSchemeFromUrl, GetUrlCookieFile, \
|
||||
ID_RE
|
||||
from error import GitError, UploadError, DownloadError
|
||||
from error import GitError, HookError, UploadError, DownloadError
|
||||
from error import ManifestInvalidRevisionError, ManifestInvalidPathError
|
||||
from error import NoManifestException
|
||||
import platform_utils
|
||||
import progress
|
||||
from repo_trace import IsTrace, Trace
|
||||
|
||||
from git_refs import GitRefs, HEAD, R_HEADS, R_TAGS, R_PUB, R_M, R_WORKTREE_M
|
||||
from git_refs import GitRefs, HEAD, R_HEADS, R_TAGS, R_PUB, R_M
|
||||
|
||||
from pyversion import is_python3
|
||||
if is_python3():
|
||||
@ -53,12 +55,6 @@ else:
|
||||
input = raw_input # noqa: F821
|
||||
|
||||
|
||||
# Maximum sleep time allowed during retries.
|
||||
MAXIMUM_RETRY_SLEEP_SEC = 3600.0
|
||||
# +-10% random jitter is added to each Fetches retry sleep duration.
|
||||
RETRY_JITTER_PERCENT = 0.1
|
||||
|
||||
|
||||
def _lwrite(path, content):
|
||||
lock = '%s.lock' % path
|
||||
|
||||
@ -205,7 +201,7 @@ class ReviewableBranch(object):
|
||||
dryrun=False,
|
||||
auto_topic=False,
|
||||
hashtags=(),
|
||||
labels=(),
|
||||
draft=False,
|
||||
private=False,
|
||||
notify=None,
|
||||
wip=False,
|
||||
@ -217,7 +213,7 @@ class ReviewableBranch(object):
|
||||
dryrun=dryrun,
|
||||
auto_topic=auto_topic,
|
||||
hashtags=hashtags,
|
||||
labels=labels,
|
||||
draft=draft,
|
||||
private=private,
|
||||
notify=notify,
|
||||
wip=wip,
|
||||
@ -403,8 +399,8 @@ class _LinkFile(object):
|
||||
else:
|
||||
src = _SafeExpandPath(self.git_worktree, self.src)
|
||||
|
||||
if not glob.has_magic(src):
|
||||
# Entity does not contain a wild card so just a simple one to one link operation.
|
||||
if os.path.exists(src):
|
||||
# Entity exists so just a simple one to one link operation.
|
||||
dest = _SafeExpandPath(self.topdir, self.dest, skipfinal=True)
|
||||
# dest & src are absolute paths at this point. Make sure the target of
|
||||
# the symlink is relative in the context of the repo client checkout.
|
||||
@ -412,7 +408,7 @@ class _LinkFile(object):
|
||||
self.__linkIt(relpath, dest)
|
||||
else:
|
||||
dest = _SafeExpandPath(self.topdir, self.dest)
|
||||
# Entity contains a wild card.
|
||||
# Entity doesn't exist assume there is a wild card
|
||||
if os.path.exists(dest) and not platform_utils.isdir(dest):
|
||||
_error('Link error: src with wildcard, %s must be a directory', dest)
|
||||
else:
|
||||
@ -449,6 +445,406 @@ class RemoteSpec(object):
|
||||
self.orig_name = orig_name
|
||||
self.fetchUrl = fetchUrl
|
||||
|
||||
|
||||
class RepoHook(object):
|
||||
|
||||
"""A RepoHook contains information about a script to run as a hook.
|
||||
|
||||
Hooks are used to run a python script before running an upload (for instance,
|
||||
to run presubmit checks). Eventually, we may have hooks for other actions.
|
||||
|
||||
This shouldn't be confused with files in the 'repo/hooks' directory. Those
|
||||
files are copied into each '.git/hooks' folder for each project. Repo-level
|
||||
hooks are associated instead with repo actions.
|
||||
|
||||
Hooks are always python. When a hook is run, we will load the hook into the
|
||||
interpreter and execute its main() function.
|
||||
"""
|
||||
|
||||
def __init__(self,
|
||||
hook_type,
|
||||
hooks_project,
|
||||
topdir,
|
||||
manifest_url,
|
||||
abort_if_user_denies=False):
|
||||
"""RepoHook constructor.
|
||||
|
||||
Params:
|
||||
hook_type: A string representing the type of hook. This is also used
|
||||
to figure out the name of the file containing the hook. For
|
||||
example: 'pre-upload'.
|
||||
hooks_project: The project containing the repo hooks. If you have a
|
||||
manifest, this is manifest.repo_hooks_project. OK if this is None,
|
||||
which will make the hook a no-op.
|
||||
topdir: Repo's top directory (the one containing the .repo directory).
|
||||
Scripts will run with CWD as this directory. If you have a manifest,
|
||||
this is manifest.topdir
|
||||
manifest_url: The URL to the manifest git repo.
|
||||
abort_if_user_denies: If True, we'll throw a HookError() if the user
|
||||
doesn't allow us to run the hook.
|
||||
"""
|
||||
self._hook_type = hook_type
|
||||
self._hooks_project = hooks_project
|
||||
self._manifest_url = manifest_url
|
||||
self._topdir = topdir
|
||||
self._abort_if_user_denies = abort_if_user_denies
|
||||
|
||||
# Store the full path to the script for convenience.
|
||||
if self._hooks_project:
|
||||
self._script_fullpath = os.path.join(self._hooks_project.worktree,
|
||||
self._hook_type + '.py')
|
||||
else:
|
||||
self._script_fullpath = None
|
||||
|
||||
def _GetHash(self):
|
||||
"""Return a hash of the contents of the hooks directory.
|
||||
|
||||
We'll just use git to do this. This hash has the property that if anything
|
||||
changes in the directory we will return a different has.
|
||||
|
||||
SECURITY CONSIDERATION:
|
||||
This hash only represents the contents of files in the hook directory, not
|
||||
any other files imported or called by hooks. Changes to imported files
|
||||
can change the script behavior without affecting the hash.
|
||||
|
||||
Returns:
|
||||
A string representing the hash. This will always be ASCII so that it can
|
||||
be printed to the user easily.
|
||||
"""
|
||||
assert self._hooks_project, "Must have hooks to calculate their hash."
|
||||
|
||||
# We will use the work_git object rather than just calling GetRevisionId().
|
||||
# That gives us a hash of the latest checked in version of the files that
|
||||
# the user will actually be executing. Specifically, GetRevisionId()
|
||||
# doesn't appear to change even if a user checks out a different version
|
||||
# of the hooks repo (via git checkout) nor if a user commits their own revs.
|
||||
#
|
||||
# NOTE: Local (non-committed) changes will not be factored into this hash.
|
||||
# I think this is OK, since we're really only worried about warning the user
|
||||
# about upstream changes.
|
||||
return self._hooks_project.work_git.rev_parse('HEAD')
|
||||
|
||||
def _GetMustVerb(self):
|
||||
"""Return 'must' if the hook is required; 'should' if not."""
|
||||
if self._abort_if_user_denies:
|
||||
return 'must'
|
||||
else:
|
||||
return 'should'
|
||||
|
||||
def _CheckForHookApproval(self):
|
||||
"""Check to see whether this hook has been approved.
|
||||
|
||||
We'll accept approval of manifest URLs if they're using secure transports.
|
||||
This way the user can say they trust the manifest hoster. For insecure
|
||||
hosts, we fall back to checking the hash of the hooks repo.
|
||||
|
||||
Note that we ask permission for each individual hook even though we use
|
||||
the hash of all hooks when detecting changes. We'd like the user to be
|
||||
able to approve / deny each hook individually. We only use the hash of all
|
||||
hooks because there is no other easy way to detect changes to local imports.
|
||||
|
||||
Returns:
|
||||
True if this hook is approved to run; False otherwise.
|
||||
|
||||
Raises:
|
||||
HookError: Raised if the user doesn't approve and abort_if_user_denies
|
||||
was passed to the consturctor.
|
||||
"""
|
||||
if self._ManifestUrlHasSecureScheme():
|
||||
return self._CheckForHookApprovalManifest()
|
||||
else:
|
||||
return self._CheckForHookApprovalHash()
|
||||
|
||||
def _CheckForHookApprovalHelper(self, subkey, new_val, main_prompt,
|
||||
changed_prompt):
|
||||
"""Check for approval for a particular attribute and hook.
|
||||
|
||||
Args:
|
||||
subkey: The git config key under [repo.hooks.<hook_type>] to store the
|
||||
last approved string.
|
||||
new_val: The new value to compare against the last approved one.
|
||||
main_prompt: Message to display to the user to ask for approval.
|
||||
changed_prompt: Message explaining why we're re-asking for approval.
|
||||
|
||||
Returns:
|
||||
True if this hook is approved to run; False otherwise.
|
||||
|
||||
Raises:
|
||||
HookError: Raised if the user doesn't approve and abort_if_user_denies
|
||||
was passed to the consturctor.
|
||||
"""
|
||||
hooks_config = self._hooks_project.config
|
||||
git_approval_key = 'repo.hooks.%s.%s' % (self._hook_type, subkey)
|
||||
|
||||
# Get the last value that the user approved for this hook; may be None.
|
||||
old_val = hooks_config.GetString(git_approval_key)
|
||||
|
||||
if old_val is not None:
|
||||
# User previously approved hook and asked not to be prompted again.
|
||||
if new_val == old_val:
|
||||
# Approval matched. We're done.
|
||||
return True
|
||||
else:
|
||||
# Give the user a reason why we're prompting, since they last told
|
||||
# us to "never ask again".
|
||||
prompt = 'WARNING: %s\n\n' % (changed_prompt,)
|
||||
else:
|
||||
prompt = ''
|
||||
|
||||
# Prompt the user if we're not on a tty; on a tty we'll assume "no".
|
||||
if sys.stdout.isatty():
|
||||
prompt += main_prompt + ' (yes/always/NO)? '
|
||||
response = input(prompt).lower()
|
||||
print()
|
||||
|
||||
# User is doing a one-time approval.
|
||||
if response in ('y', 'yes'):
|
||||
return True
|
||||
elif response == 'always':
|
||||
hooks_config.SetString(git_approval_key, new_val)
|
||||
return True
|
||||
|
||||
# For anything else, we'll assume no approval.
|
||||
if self._abort_if_user_denies:
|
||||
raise HookError('You must allow the %s hook or use --no-verify.' %
|
||||
self._hook_type)
|
||||
|
||||
return False
|
||||
|
||||
def _ManifestUrlHasSecureScheme(self):
|
||||
"""Check if the URI for the manifest is a secure transport."""
|
||||
secure_schemes = ('file', 'https', 'ssh', 'persistent-https', 'sso', 'rpc')
|
||||
parse_results = urllib.parse.urlparse(self._manifest_url)
|
||||
return parse_results.scheme in secure_schemes
|
||||
|
||||
def _CheckForHookApprovalManifest(self):
|
||||
"""Check whether the user has approved this manifest host.
|
||||
|
||||
Returns:
|
||||
True if this hook is approved to run; False otherwise.
|
||||
"""
|
||||
return self._CheckForHookApprovalHelper(
|
||||
'approvedmanifest',
|
||||
self._manifest_url,
|
||||
'Run hook scripts from %s' % (self._manifest_url,),
|
||||
'Manifest URL has changed since %s was allowed.' % (self._hook_type,))
|
||||
|
||||
def _CheckForHookApprovalHash(self):
|
||||
"""Check whether the user has approved the hooks repo.
|
||||
|
||||
Returns:
|
||||
True if this hook is approved to run; False otherwise.
|
||||
"""
|
||||
prompt = ('Repo %s run the script:\n'
|
||||
' %s\n'
|
||||
'\n'
|
||||
'Do you want to allow this script to run')
|
||||
return self._CheckForHookApprovalHelper(
|
||||
'approvedhash',
|
||||
self._GetHash(),
|
||||
prompt % (self._GetMustVerb(), self._script_fullpath),
|
||||
'Scripts have changed since %s was allowed.' % (self._hook_type,))
|
||||
|
||||
@staticmethod
|
||||
def _ExtractInterpFromShebang(data):
|
||||
"""Extract the interpreter used in the shebang.
|
||||
|
||||
Try to locate the interpreter the script is using (ignoring `env`).
|
||||
|
||||
Args:
|
||||
data: The file content of the script.
|
||||
|
||||
Returns:
|
||||
The basename of the main script interpreter, or None if a shebang is not
|
||||
used or could not be parsed out.
|
||||
"""
|
||||
firstline = data.splitlines()[:1]
|
||||
if not firstline:
|
||||
return None
|
||||
|
||||
# The format here can be tricky.
|
||||
shebang = firstline[0].strip()
|
||||
m = re.match(r'^#!\s*([^\s]+)(?:\s+([^\s]+))?', shebang)
|
||||
if not m:
|
||||
return None
|
||||
|
||||
# If the using `env`, find the target program.
|
||||
interp = m.group(1)
|
||||
if os.path.basename(interp) == 'env':
|
||||
interp = m.group(2)
|
||||
|
||||
return interp
|
||||
|
||||
def _ExecuteHookViaReexec(self, interp, context, **kwargs):
|
||||
"""Execute the hook script through |interp|.
|
||||
|
||||
Note: Support for this feature should be dropped ~Jun 2021.
|
||||
|
||||
Args:
|
||||
interp: The Python program to run.
|
||||
context: Basic Python context to execute the hook inside.
|
||||
kwargs: Arbitrary arguments to pass to the hook script.
|
||||
|
||||
Raises:
|
||||
HookError: When the hooks failed for any reason.
|
||||
"""
|
||||
# This logic needs to be kept in sync with _ExecuteHookViaImport below.
|
||||
script = """
|
||||
import json, os, sys
|
||||
path = '''%(path)s'''
|
||||
kwargs = json.loads('''%(kwargs)s''')
|
||||
context = json.loads('''%(context)s''')
|
||||
sys.path.insert(0, os.path.dirname(path))
|
||||
data = open(path).read()
|
||||
exec(compile(data, path, 'exec'), context)
|
||||
context['main'](**kwargs)
|
||||
""" % {
|
||||
'path': self._script_fullpath,
|
||||
'kwargs': json.dumps(kwargs),
|
||||
'context': json.dumps(context),
|
||||
}
|
||||
|
||||
# We pass the script via stdin to avoid OS argv limits. It also makes
|
||||
# unhandled exception tracebacks less verbose/confusing for users.
|
||||
cmd = [interp, '-c', 'import sys; exec(sys.stdin.read())']
|
||||
proc = subprocess.Popen(cmd, stdin=subprocess.PIPE)
|
||||
proc.communicate(input=script.encode('utf-8'))
|
||||
if proc.returncode:
|
||||
raise HookError('Failed to run %s hook.' % (self._hook_type,))
|
||||
|
||||
def _ExecuteHookViaImport(self, data, context, **kwargs):
|
||||
"""Execute the hook code in |data| directly.
|
||||
|
||||
Args:
|
||||
data: The code of the hook to execute.
|
||||
context: Basic Python context to execute the hook inside.
|
||||
kwargs: Arbitrary arguments to pass to the hook script.
|
||||
|
||||
Raises:
|
||||
HookError: When the hooks failed for any reason.
|
||||
"""
|
||||
# Exec, storing global context in the context dict. We catch exceptions
|
||||
# and convert to a HookError w/ just the failing traceback.
|
||||
try:
|
||||
exec(compile(data, self._script_fullpath, 'exec'), context)
|
||||
except Exception:
|
||||
raise HookError('%s\nFailed to import %s hook; see traceback above.' %
|
||||
(traceback.format_exc(), self._hook_type))
|
||||
|
||||
# Running the script should have defined a main() function.
|
||||
if 'main' not in context:
|
||||
raise HookError('Missing main() in: "%s"' % self._script_fullpath)
|
||||
|
||||
# Call the main function in the hook. If the hook should cause the
|
||||
# build to fail, it will raise an Exception. We'll catch that convert
|
||||
# to a HookError w/ just the failing traceback.
|
||||
try:
|
||||
context['main'](**kwargs)
|
||||
except Exception:
|
||||
raise HookError('%s\nFailed to run main() for %s hook; see traceback '
|
||||
'above.' % (traceback.format_exc(), self._hook_type))
|
||||
|
||||
def _ExecuteHook(self, **kwargs):
|
||||
"""Actually execute the given hook.
|
||||
|
||||
This will run the hook's 'main' function in our python interpreter.
|
||||
|
||||
Args:
|
||||
kwargs: Keyword arguments to pass to the hook. These are often specific
|
||||
to the hook type. For instance, pre-upload hooks will contain
|
||||
a project_list.
|
||||
"""
|
||||
# Keep sys.path and CWD stashed away so that we can always restore them
|
||||
# upon function exit.
|
||||
orig_path = os.getcwd()
|
||||
orig_syspath = sys.path
|
||||
|
||||
try:
|
||||
# Always run hooks with CWD as topdir.
|
||||
os.chdir(self._topdir)
|
||||
|
||||
# Put the hook dir as the first item of sys.path so hooks can do
|
||||
# relative imports. We want to replace the repo dir as [0] so
|
||||
# hooks can't import repo files.
|
||||
sys.path = [os.path.dirname(self._script_fullpath)] + sys.path[1:]
|
||||
|
||||
# Initial global context for the hook to run within.
|
||||
context = {'__file__': self._script_fullpath}
|
||||
|
||||
# Add 'hook_should_take_kwargs' to the arguments to be passed to main.
|
||||
# We don't actually want hooks to define their main with this argument--
|
||||
# it's there to remind them that their hook should always take **kwargs.
|
||||
# For instance, a pre-upload hook should be defined like:
|
||||
# def main(project_list, **kwargs):
|
||||
#
|
||||
# This allows us to later expand the API without breaking old hooks.
|
||||
kwargs = kwargs.copy()
|
||||
kwargs['hook_should_take_kwargs'] = True
|
||||
|
||||
# See what version of python the hook has been written against.
|
||||
data = open(self._script_fullpath).read()
|
||||
interp = self._ExtractInterpFromShebang(data)
|
||||
reexec = False
|
||||
if interp:
|
||||
prog = os.path.basename(interp)
|
||||
if prog.startswith('python2') and sys.version_info.major != 2:
|
||||
reexec = True
|
||||
elif prog.startswith('python3') and sys.version_info.major == 2:
|
||||
reexec = True
|
||||
|
||||
# Attempt to execute the hooks through the requested version of Python.
|
||||
if reexec:
|
||||
try:
|
||||
self._ExecuteHookViaReexec(interp, context, **kwargs)
|
||||
except OSError as e:
|
||||
if e.errno == errno.ENOENT:
|
||||
# We couldn't find the interpreter, so fallback to importing.
|
||||
reexec = False
|
||||
else:
|
||||
raise
|
||||
|
||||
# Run the hook by importing directly.
|
||||
if not reexec:
|
||||
self._ExecuteHookViaImport(data, context, **kwargs)
|
||||
finally:
|
||||
# Restore sys.path and CWD.
|
||||
sys.path = orig_syspath
|
||||
os.chdir(orig_path)
|
||||
|
||||
def Run(self, user_allows_all_hooks, **kwargs):
|
||||
"""Run the hook.
|
||||
|
||||
If the hook doesn't exist (because there is no hooks project or because
|
||||
this particular hook is not enabled), this is a no-op.
|
||||
|
||||
Args:
|
||||
user_allows_all_hooks: If True, we will never prompt about running the
|
||||
hook--we'll just assume it's OK to run it.
|
||||
kwargs: Keyword arguments to pass to the hook. These are often specific
|
||||
to the hook type. For instance, pre-upload hooks will contain
|
||||
a project_list.
|
||||
|
||||
Raises:
|
||||
HookError: If there was a problem finding the hook or the user declined
|
||||
to run a required hook (from _CheckForHookApproval).
|
||||
"""
|
||||
# No-op if there is no hooks project or if hook is disabled.
|
||||
if ((not self._hooks_project) or (self._hook_type not in
|
||||
self._hooks_project.enabled_repo_hooks)):
|
||||
return
|
||||
|
||||
# Bail with a nice error if we can't find the hook.
|
||||
if not os.path.isfile(self._script_fullpath):
|
||||
raise HookError('Couldn\'t find repo hook: "%s"' % self._script_fullpath)
|
||||
|
||||
# Make sure the user is OK with running the hook.
|
||||
if (not user_allows_all_hooks) and (not self._CheckForHookApproval()):
|
||||
return
|
||||
|
||||
# Run the hook with the same version of python we're using.
|
||||
self._ExecuteHook(**kwargs)
|
||||
|
||||
|
||||
class Project(object):
|
||||
# These objects can be shared between several working trees.
|
||||
shareable_files = ['description', 'info']
|
||||
@ -479,7 +875,6 @@ class Project(object):
|
||||
is_derived=False,
|
||||
dest_branch=None,
|
||||
optimized_fetch=False,
|
||||
retry_fetches=0,
|
||||
old_revision=None):
|
||||
"""Init a Project object.
|
||||
|
||||
@ -506,8 +901,6 @@ class Project(object):
|
||||
dest_branch: The branch to which to push changes for review by default.
|
||||
optimized_fetch: If True, when a project is set to a sha1 revision, only
|
||||
fetch from the remote if the sha1 is not present locally.
|
||||
retry_fetches: Retry remote fetches n times upon receiving transient error
|
||||
with exponential backoff and jitter.
|
||||
old_revision: saved git commit id for open GITC projects.
|
||||
"""
|
||||
self.manifest = manifest
|
||||
@ -543,7 +936,6 @@ class Project(object):
|
||||
self.use_git_worktrees = use_git_worktrees
|
||||
self.is_derived = is_derived
|
||||
self.optimized_fetch = optimized_fetch
|
||||
self.retry_fetches = max(0, retry_fetches)
|
||||
self.subprojects = []
|
||||
|
||||
self.snapshots = {}
|
||||
@ -954,7 +1346,7 @@ class Project(object):
|
||||
dryrun=False,
|
||||
auto_topic=False,
|
||||
hashtags=(),
|
||||
labels=(),
|
||||
draft=False,
|
||||
private=False,
|
||||
notify=None,
|
||||
wip=False,
|
||||
@ -1004,12 +1396,16 @@ class Project(object):
|
||||
if dest_branch.startswith(R_HEADS):
|
||||
dest_branch = dest_branch[len(R_HEADS):]
|
||||
|
||||
ref_spec = '%s:refs/for/%s' % (R_HEADS + branch.name, dest_branch)
|
||||
upload_type = 'for'
|
||||
if draft:
|
||||
upload_type = 'drafts'
|
||||
|
||||
ref_spec = '%s:refs/%s/%s' % (R_HEADS + branch.name, upload_type,
|
||||
dest_branch)
|
||||
opts = []
|
||||
if auto_topic:
|
||||
opts += ['topic=' + branch.name]
|
||||
opts += ['t=%s' % p for p in hashtags]
|
||||
opts += ['l=%s' % p for p in labels]
|
||||
|
||||
opts += ['r=%s' % p for p in people[0]]
|
||||
opts += ['cc=%s' % p for p in people[1]]
|
||||
@ -1057,7 +1453,6 @@ class Project(object):
|
||||
tags=True,
|
||||
archive=False,
|
||||
optimized_fetch=False,
|
||||
retry_fetches=0,
|
||||
prune=False,
|
||||
submodules=False,
|
||||
clone_filter=None):
|
||||
@ -1141,7 +1536,7 @@ class Project(object):
|
||||
current_branch_only=current_branch_only,
|
||||
tags=tags, prune=prune, depth=depth,
|
||||
submodules=submodules, force_sync=force_sync,
|
||||
clone_filter=clone_filter, retry_fetches=retry_fetches):
|
||||
clone_filter=clone_filter):
|
||||
return False
|
||||
|
||||
mp = self.manifest.manifestProject
|
||||
@ -1909,27 +2304,6 @@ class Project(object):
|
||||
# Enable the extension!
|
||||
self.config.SetString('extensions.%s' % (key,), value)
|
||||
|
||||
def ResolveRemoteHead(self, name=None):
|
||||
"""Find out what the default branch (HEAD) points to.
|
||||
|
||||
Normally this points to refs/heads/master, but projects are moving to main.
|
||||
Support whatever the server uses rather than hardcoding "master" ourselves.
|
||||
"""
|
||||
if name is None:
|
||||
name = self.remote.name
|
||||
|
||||
# The output will look like (NB: tabs are separators):
|
||||
# ref: refs/heads/master HEAD
|
||||
# 5f6803b100bb3cd0f534e96e88c91373e8ed1c44 HEAD
|
||||
output = self.bare_git.ls_remote('-q', '--symref', '--exit-code', name, 'HEAD')
|
||||
|
||||
for line in output.splitlines():
|
||||
lhs, rhs = line.split('\t', 1)
|
||||
if rhs == 'HEAD' and lhs.startswith('ref:'):
|
||||
return lhs[4:].strip()
|
||||
|
||||
return None
|
||||
|
||||
def _CheckForImmutableRevision(self):
|
||||
try:
|
||||
# if revision (sha or tag) is not present then following function
|
||||
@ -1964,10 +2338,8 @@ class Project(object):
|
||||
depth=None,
|
||||
submodules=False,
|
||||
force_sync=False,
|
||||
clone_filter=None,
|
||||
retry_fetches=2,
|
||||
retry_sleep_initial_sec=4.0,
|
||||
retry_exp_factor=2.0):
|
||||
clone_filter=None):
|
||||
|
||||
is_sha1 = False
|
||||
tag_name = None
|
||||
# The depth should not be used when fetching to a mirror because
|
||||
@ -2072,10 +2444,8 @@ class Project(object):
|
||||
if os.path.exists(os.path.join(self.gitdir, 'shallow')):
|
||||
cmd.append('--depth=2147483647')
|
||||
|
||||
if not verbose:
|
||||
if quiet:
|
||||
cmd.append('--quiet')
|
||||
if not quiet and sys.stdout.isatty():
|
||||
cmd.append('--progress')
|
||||
if not self.worktree:
|
||||
cmd.append('--update-head-ok')
|
||||
cmd.append(name)
|
||||
@ -2129,37 +2499,18 @@ class Project(object):
|
||||
|
||||
cmd.extend(spec)
|
||||
|
||||
# At least one retry minimum due to git remote prune.
|
||||
retry_fetches = max(retry_fetches, 2)
|
||||
retry_cur_sleep = retry_sleep_initial_sec
|
||||
ok = prune_tried = False
|
||||
for try_n in range(retry_fetches):
|
||||
ok = False
|
||||
for _i in range(2):
|
||||
gitcmd = GitCommand(self, cmd, bare=True, ssh_proxy=ssh_proxy,
|
||||
merge_output=True, capture_stdout=quiet)
|
||||
merge_output=True, capture_stdout=not verbose)
|
||||
ret = gitcmd.Wait()
|
||||
if ret == 0:
|
||||
ok = True
|
||||
break
|
||||
|
||||
# Retry later due to HTTP 429 Too Many Requests.
|
||||
elif ('error:' in gitcmd.stderr and
|
||||
'HTTP 429' in gitcmd.stderr):
|
||||
if not quiet:
|
||||
print('429 received, sleeping: %s sec' % retry_cur_sleep,
|
||||
file=sys.stderr)
|
||||
time.sleep(retry_cur_sleep)
|
||||
retry_cur_sleep = min(retry_exp_factor * retry_cur_sleep,
|
||||
MAXIMUM_RETRY_SLEEP_SEC)
|
||||
retry_cur_sleep *= (1 - random.uniform(-RETRY_JITTER_PERCENT,
|
||||
RETRY_JITTER_PERCENT))
|
||||
continue
|
||||
|
||||
# If this is not last attempt, try 'git remote prune'.
|
||||
elif (try_n < retry_fetches - 1 and
|
||||
'error:' in gitcmd.stderr and
|
||||
'git remote prune' in gitcmd.stderr and
|
||||
not prune_tried):
|
||||
prune_tried = True
|
||||
# If needed, run the 'git remote prune' the first time through the loop
|
||||
elif (not _i and
|
||||
"error:" in gitcmd.stderr and
|
||||
"git remote prune" in gitcmd.stderr):
|
||||
prunecmd = GitCommand(self, ['remote', 'prune', name], bare=True,
|
||||
ssh_proxy=ssh_proxy)
|
||||
ret = prunecmd.Wait()
|
||||
@ -2231,10 +2582,8 @@ class Project(object):
|
||||
return False
|
||||
|
||||
cmd = ['fetch']
|
||||
if not verbose:
|
||||
if quiet:
|
||||
cmd.append('--quiet')
|
||||
if not quiet and sys.stdout.isatty():
|
||||
cmd.append('--progress')
|
||||
if not self.worktree:
|
||||
cmd.append('--update-head-ok')
|
||||
cmd.append(bundle_dst)
|
||||
@ -2294,10 +2643,9 @@ class Project(object):
|
||||
# 22: HTTP page not retrieved. The requested url was not found or
|
||||
# returned another error with the HTTP error code being 400 or above.
|
||||
# This return code only appears if -f, --fail is used.
|
||||
if verbose:
|
||||
print('%s: Unable to retrieve clone.bundle; ignoring.' % self.name)
|
||||
if output:
|
||||
print('Curl output:\n%s' % output)
|
||||
if not quiet:
|
||||
print("Server does not provide clone.bundle; ignoring.",
|
||||
file=sys.stderr)
|
||||
return False
|
||||
elif curlret and not verbose and output:
|
||||
print('%s' % output, file=sys.stderr)
|
||||
@ -2334,12 +2682,8 @@ class Project(object):
|
||||
if self._allrefs:
|
||||
raise GitError('%s checkout %s ' % (self.name, rev))
|
||||
|
||||
def _CherryPick(self, rev, ffonly=False, record_origin=False):
|
||||
def _CherryPick(self, rev):
|
||||
cmd = ['cherry-pick']
|
||||
if ffonly:
|
||||
cmd.append('--ff')
|
||||
if record_origin:
|
||||
cmd.append('-x')
|
||||
cmd.append(rev)
|
||||
cmd.append('--')
|
||||
if GitCommand(self, cmd).Wait() != 0:
|
||||
@ -2401,19 +2745,10 @@ class Project(object):
|
||||
os.makedirs(self.objdir)
|
||||
self.bare_objdir.init()
|
||||
|
||||
if self.use_git_worktrees:
|
||||
# Set up the m/ space to point to the worktree-specific ref space.
|
||||
# We'll update the worktree-specific ref space on each checkout.
|
||||
if self.manifest.branch:
|
||||
self.bare_git.symbolic_ref(
|
||||
'-m', 'redirecting to worktree scope',
|
||||
R_M + self.manifest.branch,
|
||||
R_WORKTREE_M + self.manifest.branch)
|
||||
|
||||
# Enable per-worktree config file support if possible. This is more a
|
||||
# nice-to-have feature for users rather than a hard requirement.
|
||||
if git_require((2, 20, 0)):
|
||||
self.EnableRepositoryExtension('worktreeConfig')
|
||||
# Enable per-worktree config file support if possible. This is more a
|
||||
# nice-to-have feature for users rather than a hard requirement.
|
||||
if self.use_git_worktrees and git_require((2, 19, 0)):
|
||||
self.EnableRepositoryExtension('worktreeConfig')
|
||||
|
||||
# If we have a separate directory to hold refs, initialize it as well.
|
||||
if self.objdir != self.gitdir:
|
||||
@ -2548,37 +2883,25 @@ class Project(object):
|
||||
|
||||
def _InitMRef(self):
|
||||
if self.manifest.branch:
|
||||
if self.use_git_worktrees:
|
||||
# We can't update this ref with git worktrees until it exists.
|
||||
# We'll wait until the initial checkout to set it.
|
||||
if not os.path.exists(self.worktree):
|
||||
return
|
||||
|
||||
base = R_WORKTREE_M
|
||||
active_git = self.work_git
|
||||
else:
|
||||
base = R_M
|
||||
active_git = self.bare_git
|
||||
|
||||
self._InitAnyMRef(base + self.manifest.branch, active_git)
|
||||
self._InitAnyMRef(R_M + self.manifest.branch)
|
||||
|
||||
def _InitMirrorHead(self):
|
||||
self._InitAnyMRef(HEAD, self.bare_git)
|
||||
self._InitAnyMRef(HEAD)
|
||||
|
||||
def _InitAnyMRef(self, ref, active_git):
|
||||
def _InitAnyMRef(self, ref):
|
||||
cur = self.bare_ref.symref(ref)
|
||||
|
||||
if self.revisionId:
|
||||
if cur != '' or self.bare_ref.get(ref) != self.revisionId:
|
||||
msg = 'manifest set to %s' % self.revisionId
|
||||
dst = self.revisionId + '^0'
|
||||
active_git.UpdateRef(ref, dst, message=msg, detach=True)
|
||||
self.bare_git.UpdateRef(ref, dst, message=msg, detach=True)
|
||||
else:
|
||||
remote = self.GetRemote(self.remote.name)
|
||||
dst = remote.ToLocal(self.revisionExpr)
|
||||
if cur != dst:
|
||||
msg = 'manifest set to %s' % self.revisionExpr
|
||||
active_git.symbolic_ref('-m', msg, ref, dst)
|
||||
self.bare_git.symbolic_ref('-m', msg, ref, dst)
|
||||
|
||||
def _CheckDirReference(self, srcdir, destdir, share_refs):
|
||||
# Git worktrees don't use symlinks to share at all.
|
||||
@ -2709,8 +3032,6 @@ class Project(object):
|
||||
with open(os.path.join(git_worktree_path, 'gitdir'), 'w') as fp:
|
||||
print(os.path.relpath(dotgit, git_worktree_path), file=fp)
|
||||
|
||||
self._InitMRef()
|
||||
|
||||
def _InitWorkTree(self, force_sync=False, submodules=False):
|
||||
realdotgit = os.path.join(self.worktree, '.git')
|
||||
tmpdotgit = realdotgit + '.tmp'
|
||||
@ -2827,13 +3148,6 @@ class Project(object):
|
||||
self._bare = bare
|
||||
self._gitdir = gitdir
|
||||
|
||||
# __getstate__ and __setstate__ are required for pickling because __getattr__ exists.
|
||||
def __getstate__(self):
|
||||
return (self._project, self._bare, self._gitdir)
|
||||
|
||||
def __setstate__(self, state):
|
||||
self._project, self._bare, self._gitdir = state
|
||||
|
||||
def LsOthers(self):
|
||||
p = GitCommand(self._project,
|
||||
['ls-files',
|
||||
|
||||
@ -15,10 +15,7 @@
|
||||
|
||||
"""Helper tool for signing repo release tags correctly.
|
||||
|
||||
This is intended to be run only by the official Repo release managers, but it
|
||||
could be run by people maintaining their own fork of the project.
|
||||
|
||||
NB: Check docs/release-process.md for production freeze information.
|
||||
This is intended to be run only by the official Repo release managers.
|
||||
"""
|
||||
|
||||
import argparse
|
||||
@ -89,9 +86,7 @@ To roll back a release:
|
||||
|
||||
def get_parser():
|
||||
"""Get a CLI parser."""
|
||||
parser = argparse.ArgumentParser(
|
||||
description=__doc__,
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument('-n', '--dry-run',
|
||||
dest='dryrun', action='store_true',
|
||||
help='show everything that would be done')
|
||||
|
||||
208
repo
208
repo
@ -133,7 +133,7 @@ if not REPO_REV:
|
||||
REPO_REV = 'stable'
|
||||
|
||||
# increment this whenever we make important changes to this script
|
||||
VERSION = (2, 8)
|
||||
VERSION = (2, 4)
|
||||
|
||||
# increment this if the MAINTAINER_KEYS block is modified
|
||||
KEYRING_VERSION = (2, 3)
|
||||
@ -251,6 +251,8 @@ else:
|
||||
home_dot_repo = os.path.expanduser('~/.repoconfig')
|
||||
gpg_dir = os.path.join(home_dot_repo, 'gnupg')
|
||||
|
||||
extra_args = []
|
||||
|
||||
|
||||
def GetParser(gitc_init=False):
|
||||
"""Setup the CLI parser."""
|
||||
@ -317,11 +319,9 @@ def GetParser(gitc_init=False):
|
||||
help='restrict manifest projects to ones with a specified '
|
||||
'platform group [auto|all|none|linux|darwin|...]',
|
||||
metavar='PLATFORM')
|
||||
group.add_option('--clone-bundle', action='store_true',
|
||||
help='enable use of /clone.bundle on HTTP/HTTPS (default if not --partial-clone)')
|
||||
group.add_option('--no-clone-bundle',
|
||||
dest='clone_bundle', action='store_false',
|
||||
help='disable use of /clone.bundle on HTTP/HTTPS (default if --partial-clone)')
|
||||
dest='clone_bundle', default=True, action='store_false',
|
||||
help='disable use of /clone.bundle on HTTP/HTTPS')
|
||||
group.add_option('--no-tags',
|
||||
dest='tags', default=True, action='store_false',
|
||||
help="don't fetch tags in the manifest")
|
||||
@ -330,10 +330,8 @@ def GetParser(gitc_init=False):
|
||||
group = parser.add_option_group('repo Version options')
|
||||
group.add_option('--repo-url', metavar='URL',
|
||||
help='repo repository location ($REPO_URL)')
|
||||
group.add_option('--repo-rev', metavar='REV',
|
||||
group.add_option('--repo-branch', metavar='REVISION',
|
||||
help='repo branch or revision ($REPO_REV)')
|
||||
group.add_option('--repo-branch', dest='repo_rev',
|
||||
help=optparse.SUPPRESS_HELP)
|
||||
group.add_option('--no-repo-verify',
|
||||
dest='repo_verify', default=True, action='store_false',
|
||||
help='do not verify repo source code')
|
||||
@ -465,34 +463,6 @@ class CloneFailure(Exception):
|
||||
"""
|
||||
|
||||
|
||||
def check_repo_verify(repo_verify, quiet=False):
|
||||
"""Check the --repo-verify state."""
|
||||
if not repo_verify:
|
||||
print('repo: warning: verification of repo code has been disabled;\n'
|
||||
'repo will not be able to verify the integrity of itself.\n',
|
||||
file=sys.stderr)
|
||||
return False
|
||||
|
||||
if NeedSetupGnuPG():
|
||||
return SetupGnuPG(quiet)
|
||||
|
||||
return True
|
||||
|
||||
|
||||
def check_repo_rev(dst, rev, repo_verify=True, quiet=False):
|
||||
"""Check that |rev| is valid."""
|
||||
do_verify = check_repo_verify(repo_verify, quiet=quiet)
|
||||
remote_ref, local_rev = resolve_repo_rev(dst, rev)
|
||||
if not quiet and not remote_ref.startswith('refs/heads/'):
|
||||
print('warning: repo is not tracking a remote branch, so it will not '
|
||||
'receive updates', file=sys.stderr)
|
||||
if do_verify:
|
||||
rev = verify_rev(dst, remote_ref, local_rev, quiet)
|
||||
else:
|
||||
rev = local_rev
|
||||
return (remote_ref, rev)
|
||||
|
||||
|
||||
def _Init(args, gitc_init=False):
|
||||
"""Installs repo by cloning it over the network.
|
||||
"""
|
||||
@ -504,11 +474,21 @@ def _Init(args, gitc_init=False):
|
||||
opt.quiet = opt.output_mode is False
|
||||
opt.verbose = opt.output_mode is True
|
||||
|
||||
if opt.clone_bundle is None:
|
||||
opt.clone_bundle = False if opt.partial_clone else True
|
||||
url = opt.repo_url
|
||||
if not url:
|
||||
url = REPO_URL
|
||||
extra_args.append('--repo-url=%s' % url)
|
||||
|
||||
url = opt.repo_url or REPO_URL
|
||||
rev = opt.repo_rev or REPO_REV
|
||||
branch = opt.repo_branch
|
||||
if not branch:
|
||||
branch = REPO_REV
|
||||
extra_args.append('--repo-branch=%s' % branch)
|
||||
|
||||
if branch.startswith('refs/heads/'):
|
||||
branch = branch[len('refs/heads/'):]
|
||||
if branch.startswith('refs/'):
|
||||
print("fatal: invalid branch name '%s'" % branch, file=sys.stderr)
|
||||
raise CloneFailure()
|
||||
|
||||
try:
|
||||
if gitc_init:
|
||||
@ -543,13 +523,25 @@ def _Init(args, gitc_init=False):
|
||||
|
||||
_CheckGitVersion()
|
||||
try:
|
||||
if not opt.repo_verify:
|
||||
do_verify = False
|
||||
else:
|
||||
if NeedSetupGnuPG():
|
||||
do_verify = SetupGnuPG(opt.quiet)
|
||||
else:
|
||||
do_verify = True
|
||||
|
||||
if not opt.quiet:
|
||||
print('Downloading Repo source from', url)
|
||||
dst = os.path.abspath(os.path.join(repodir, S_repo))
|
||||
_Clone(url, dst, opt.clone_bundle, opt.quiet, opt.verbose)
|
||||
|
||||
remote_ref, rev = check_repo_rev(dst, rev, opt.repo_verify, quiet=opt.quiet)
|
||||
_Checkout(dst, remote_ref, rev, opt.quiet)
|
||||
if do_verify:
|
||||
rev = _Verify(dst, branch, opt.quiet)
|
||||
else:
|
||||
rev = 'refs/remotes/origin/%s^0' % branch
|
||||
|
||||
_Checkout(dst, branch, rev, opt.quiet)
|
||||
|
||||
if not os.path.isfile(os.path.join(dst, 'repo')):
|
||||
print("warning: '%s' does not look like a git-repo repository, is "
|
||||
@ -614,8 +606,7 @@ def _CheckGitVersion():
|
||||
|
||||
if ver_act < MIN_GIT_VERSION:
|
||||
need = '.'.join(map(str, MIN_GIT_VERSION))
|
||||
print('fatal: git %s or later required; found %s' % (need, ver_act.full),
|
||||
file=sys.stderr)
|
||||
print('fatal: git %s or later required' % need, file=sys.stderr)
|
||||
raise CloneFailure()
|
||||
|
||||
|
||||
@ -764,17 +755,15 @@ def _InitHttp():
|
||||
|
||||
def _Fetch(url, cwd, src, quiet, verbose):
|
||||
cmd = ['fetch']
|
||||
if not verbose:
|
||||
if quiet:
|
||||
cmd.append('--quiet')
|
||||
err = None
|
||||
if not quiet and sys.stdout.isatty():
|
||||
cmd.append('--progress')
|
||||
elif not verbose:
|
||||
err = subprocess.PIPE
|
||||
else:
|
||||
err = None
|
||||
cmd.append(src)
|
||||
cmd.append('+refs/heads/*:refs/remotes/origin/*')
|
||||
cmd.append('+refs/tags/*:refs/tags/*')
|
||||
run_git(*cmd, stderr=err, capture_output=False, cwd=cwd)
|
||||
run_git(*cmd, stderr=err, cwd=cwd)
|
||||
|
||||
|
||||
def _DownloadBundle(url, cwd, quiet, verbose):
|
||||
@ -857,83 +846,23 @@ def _Clone(url, cwd, clone_bundle, quiet, verbose):
|
||||
_Fetch(url, cwd, 'origin', quiet, verbose)
|
||||
|
||||
|
||||
def resolve_repo_rev(cwd, committish):
|
||||
"""Figure out what REPO_REV represents.
|
||||
|
||||
We support:
|
||||
* refs/heads/xxx: Branch.
|
||||
* refs/tags/xxx: Tag.
|
||||
* xxx: Branch or tag or commit.
|
||||
|
||||
Args:
|
||||
cwd: The git checkout to run in.
|
||||
committish: The REPO_REV argument to resolve.
|
||||
|
||||
Returns:
|
||||
A tuple of (remote ref, commit) as makes sense for the committish.
|
||||
For branches, this will look like ('refs/heads/stable', <revision>).
|
||||
For tags, this will look like ('refs/tags/v1.0', <revision>).
|
||||
For commits, this will be (<revision>, <revision>).
|
||||
def _Verify(cwd, branch, quiet):
|
||||
"""Verify the branch has been signed by a tag.
|
||||
"""
|
||||
def resolve(committish):
|
||||
ret = run_git('rev-parse', '--verify', '%s^{commit}' % (committish,),
|
||||
cwd=cwd, check=False)
|
||||
return None if ret.returncode else ret.stdout.strip()
|
||||
|
||||
# An explicit branch.
|
||||
if committish.startswith('refs/heads/'):
|
||||
remote_ref = committish
|
||||
committish = committish[len('refs/heads/'):]
|
||||
rev = resolve('refs/remotes/origin/%s' % committish)
|
||||
if rev is None:
|
||||
print('repo: error: unknown branch "%s"' % (committish,),
|
||||
file=sys.stderr)
|
||||
raise CloneFailure()
|
||||
return (remote_ref, rev)
|
||||
|
||||
# An explicit tag.
|
||||
if committish.startswith('refs/tags/'):
|
||||
remote_ref = committish
|
||||
committish = committish[len('refs/tags/'):]
|
||||
rev = resolve(remote_ref)
|
||||
if rev is None:
|
||||
print('repo: error: unknown tag "%s"' % (committish,),
|
||||
file=sys.stderr)
|
||||
raise CloneFailure()
|
||||
return (remote_ref, rev)
|
||||
|
||||
# See if it's a short branch name.
|
||||
rev = resolve('refs/remotes/origin/%s' % committish)
|
||||
if rev:
|
||||
return ('refs/heads/%s' % (committish,), rev)
|
||||
|
||||
# See if it's a tag.
|
||||
rev = resolve('refs/tags/%s' % committish)
|
||||
if rev:
|
||||
return ('refs/tags/%s' % (committish,), rev)
|
||||
|
||||
# See if it's a commit.
|
||||
rev = resolve(committish)
|
||||
if rev and rev.lower().startswith(committish.lower()):
|
||||
return (rev, rev)
|
||||
|
||||
# Give up!
|
||||
print('repo: error: unable to resolve "%s"' % (committish,), file=sys.stderr)
|
||||
raise CloneFailure()
|
||||
|
||||
|
||||
def verify_rev(cwd, remote_ref, rev, quiet):
|
||||
"""Verify the commit has been signed by a tag."""
|
||||
ret = run_git('describe', rev, cwd=cwd)
|
||||
cur = ret.stdout.strip()
|
||||
try:
|
||||
ret = run_git('describe', 'origin/%s' % branch, cwd=cwd)
|
||||
cur = ret.stdout.strip()
|
||||
except CloneFailure:
|
||||
print("fatal: branch '%s' has not been signed" % branch, file=sys.stderr)
|
||||
raise
|
||||
|
||||
m = re.compile(r'^(.*)-[0-9]{1,}-g[0-9a-f]{1,}$').match(cur)
|
||||
if m:
|
||||
cur = m.group(1)
|
||||
if not quiet:
|
||||
print(file=sys.stderr)
|
||||
print("warning: '%s' is not signed; falling back to signed release '%s'"
|
||||
% (remote_ref, cur), file=sys.stderr)
|
||||
print("info: Ignoring branch '%s'; using tagged release '%s'"
|
||||
% (branch, cur), file=sys.stderr)
|
||||
print(file=sys.stderr)
|
||||
|
||||
env = os.environ.copy()
|
||||
@ -942,13 +871,13 @@ def verify_rev(cwd, remote_ref, rev, quiet):
|
||||
return '%s^0' % cur
|
||||
|
||||
|
||||
def _Checkout(cwd, remote_ref, rev, quiet):
|
||||
def _Checkout(cwd, branch, rev, quiet):
|
||||
"""Checkout an upstream branch into the repository and track it.
|
||||
"""
|
||||
run_git('update-ref', 'refs/heads/default', rev, cwd=cwd)
|
||||
|
||||
_SetConfig(cwd, 'branch.default.remote', 'origin')
|
||||
_SetConfig(cwd, 'branch.default.merge', remote_ref)
|
||||
_SetConfig(cwd, 'branch.default.merge', 'refs/heads/%s' % branch)
|
||||
|
||||
run_git('symbolic-ref', 'HEAD', 'refs/heads/default', cwd=cwd)
|
||||
|
||||
@ -966,7 +895,9 @@ def _FindRepo():
|
||||
repo = None
|
||||
|
||||
olddir = None
|
||||
while curdir != olddir and not repo:
|
||||
while curdir != '/' \
|
||||
and curdir != olddir \
|
||||
and not repo:
|
||||
repo = os.path.join(curdir, repodir, REPO_MAIN)
|
||||
if not os.path.isfile(repo):
|
||||
repo = None
|
||||
@ -1062,14 +993,6 @@ def _Version():
|
||||
print(' (from %s)' % (__file__,))
|
||||
print('git %s' % (ParseGitVersion().full,))
|
||||
print('Python %s' % sys.version)
|
||||
uname = platform.uname()
|
||||
if sys.version_info.major < 3:
|
||||
# Python 3 returns a named tuple, but Python 2 is simpler.
|
||||
print(uname)
|
||||
else:
|
||||
print('OS %s %s (%s)' % (uname.system, uname.release, uname.version))
|
||||
print('CPU %s (%s)' %
|
||||
(uname.machine, uname.processor if uname.processor else 'unknown'))
|
||||
sys.exit(0)
|
||||
|
||||
|
||||
@ -1105,18 +1028,12 @@ def _SetDefaultsTo(gitdir):
|
||||
global REPO_REV
|
||||
|
||||
REPO_URL = gitdir
|
||||
ret = run_git('--git-dir=%s' % gitdir, 'symbolic-ref', 'HEAD', check=False)
|
||||
if ret.returncode:
|
||||
# If we're not tracking a branch (bisect/etc...), then fall back to commit.
|
||||
print('repo: warning: %s has no current branch; using HEAD' % gitdir,
|
||||
file=sys.stderr)
|
||||
try:
|
||||
ret = run_git('rev-parse', 'HEAD', cwd=gitdir)
|
||||
except CloneFailure:
|
||||
print('fatal: %s has invalid HEAD' % gitdir, file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
REPO_REV = ret.stdout.strip()
|
||||
try:
|
||||
ret = run_git('--git-dir=%s' % gitdir, 'symbolic-ref', 'HEAD')
|
||||
REPO_REV = ret.stdout.strip()
|
||||
except CloneFailure:
|
||||
print('fatal: %s has no current branch' % gitdir, file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
def main(orig_args):
|
||||
@ -1172,10 +1089,6 @@ def main(orig_args):
|
||||
if my_main:
|
||||
repo_main = my_main
|
||||
|
||||
if not repo_main:
|
||||
print("fatal: unable to find repo entry point", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
ver_str = '.'.join(map(str, VERSION))
|
||||
me = [sys.executable, repo_main,
|
||||
'--repo-dir=%s' % rel_repo_dir,
|
||||
@ -1183,6 +1096,7 @@ def main(orig_args):
|
||||
'--wrapper-path=%s' % wrapper_path,
|
||||
'--']
|
||||
me.extend(orig_args)
|
||||
me.extend(extra_args)
|
||||
exec_command(me)
|
||||
print("fatal: unable to start %s" % repo_main, file=sys.stderr)
|
||||
sys.exit(148)
|
||||
|
||||
@ -16,7 +16,6 @@
|
||||
|
||||
import os
|
||||
|
||||
# A mapping of the subcommand name to the class that implements it.
|
||||
all_commands = {}
|
||||
|
||||
my_dir = os.path.dirname(__file__)
|
||||
@ -38,7 +37,7 @@ for py in os.listdir(my_dir):
|
||||
['%s' % name])
|
||||
mod = getattr(mod, name)
|
||||
try:
|
||||
cmd = getattr(mod, clsn)
|
||||
cmd = getattr(mod, clsn)()
|
||||
except AttributeError:
|
||||
raise SyntaxError('%s/%s does not define class %s' % (
|
||||
__name__, py, clsn))
|
||||
@ -47,5 +46,5 @@ for py in os.listdir(my_dir):
|
||||
cmd.NAME = name
|
||||
all_commands[name] = cmd
|
||||
|
||||
# Add 'branch' as an alias for 'branches'.
|
||||
all_commands['branch'] = all_commands['branches']
|
||||
if 'help' in all_commands:
|
||||
all_commands['help'].commands = all_commands
|
||||
|
||||
@ -79,7 +79,7 @@ synced and their revisions won't be found.
|
||||
metavar='<FORMAT>',
|
||||
help='print the log using a custom git pretty format string')
|
||||
|
||||
def _printRawDiff(self, diff, pretty_format=None):
|
||||
def _printRawDiff(self, diff):
|
||||
for project in diff['added']:
|
||||
self.printText("A %s %s" % (project.relpath, project.revisionExpr))
|
||||
self.out.nl()
|
||||
@ -92,7 +92,7 @@ synced and their revisions won't be found.
|
||||
self.printText("C %s %s %s" % (project.relpath, project.revisionExpr,
|
||||
otherProject.revisionExpr))
|
||||
self.out.nl()
|
||||
self._printLogs(project, otherProject, raw=True, color=False, pretty_format=pretty_format)
|
||||
self._printLogs(project, otherProject, raw=True, color=False)
|
||||
|
||||
for project, otherProject in diff['unreachable']:
|
||||
self.printText("U %s %s %s" % (project.relpath, project.revisionExpr,
|
||||
@ -203,6 +203,6 @@ synced and their revisions won't be found.
|
||||
|
||||
diff = manifest1.projectsDiff(manifest2)
|
||||
if opt.raw:
|
||||
self._printRawDiff(diff, pretty_format=opt.pretty_format)
|
||||
self._printRawDiff(diff)
|
||||
else:
|
||||
self._printDiff(diff, color=opt.color, pretty_format=opt.pretty_format)
|
||||
|
||||
@ -37,13 +37,9 @@ If no project is specified try to use current directory as a project.
|
||||
"""
|
||||
|
||||
def _Options(self, p):
|
||||
p.add_option('-b', '--branch',
|
||||
help='create a new branch first')
|
||||
p.add_option('-c', '--cherry-pick',
|
||||
dest='cherrypick', action='store_true',
|
||||
help="cherry-pick instead of checkout")
|
||||
p.add_option('-x', '--record-origin', action='store_true',
|
||||
help='pass -x when cherry-picking')
|
||||
p.add_option('-r', '--revert',
|
||||
dest='revert', action='store_true',
|
||||
help="revert instead of checkout")
|
||||
@ -82,14 +78,6 @@ If no project is specified try to use current directory as a project.
|
||||
project = self.GetProjects([a])[0]
|
||||
return to_get
|
||||
|
||||
def ValidateOptions(self, opt, args):
|
||||
if opt.record_origin:
|
||||
if not opt.cherrypick:
|
||||
self.OptionParser.error('-x only makes sense with --cherry-pick')
|
||||
|
||||
if opt.ffonly:
|
||||
self.OptionParser.error('-x and --ff are mutually exclusive options')
|
||||
|
||||
def Execute(self, opt, args):
|
||||
for project, change_id, ps_id in self._ParseChangeIds(args):
|
||||
dl = project.DownloadPatchSet(change_id, ps_id)
|
||||
@ -111,36 +99,17 @@ If no project is specified try to use current directory as a project.
|
||||
file=sys.stderr)
|
||||
for c in dl.commits:
|
||||
print(' %s' % (c), file=sys.stderr)
|
||||
|
||||
if opt.cherrypick:
|
||||
mode = 'cherry-pick'
|
||||
try:
|
||||
project._CherryPick(dl.commit)
|
||||
except GitError:
|
||||
print('[%s] Could not complete the cherry-pick of %s'
|
||||
% (project.name, dl.commit), file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
elif opt.revert:
|
||||
mode = 'revert'
|
||||
project._Revert(dl.commit)
|
||||
elif opt.ffonly:
|
||||
mode = 'fast-forward merge'
|
||||
project._FastForward(dl.commit, ffonly=True)
|
||||
else:
|
||||
mode = 'checkout'
|
||||
|
||||
# We'll combine the branch+checkout operation, but all the rest need a
|
||||
# dedicated branch start.
|
||||
if opt.branch and mode != 'checkout':
|
||||
project.StartBranch(opt.branch)
|
||||
|
||||
try:
|
||||
if opt.cherrypick:
|
||||
project._CherryPick(dl.commit, ffonly=opt.ffonly,
|
||||
record_origin=opt.record_origin)
|
||||
elif opt.revert:
|
||||
project._Revert(dl.commit)
|
||||
elif opt.ffonly:
|
||||
project._FastForward(dl.commit, ffonly=True)
|
||||
else:
|
||||
if opt.branch:
|
||||
project.StartBranch(opt.branch, revision=dl.commit)
|
||||
else:
|
||||
project._Checkout(dl.commit)
|
||||
|
||||
except GitError:
|
||||
print('[%s] Could not complete the %s of %s'
|
||||
% (project.name, mode, dl.commit), file=sys.stderr)
|
||||
sys.exit(1)
|
||||
project._Checkout(dl.commit)
|
||||
|
||||
@ -179,8 +179,6 @@ without iterating through the remaining projects.
|
||||
'annotations': dict((a.name, a.value) for a in project.annotations),
|
||||
'gitdir': project.gitdir,
|
||||
'worktree': project.worktree,
|
||||
'upstream': project.upstream,
|
||||
'dest_branch': project.dest_branch,
|
||||
}
|
||||
|
||||
def ValidateOptions(self, opt, args):
|
||||
@ -319,8 +317,6 @@ def DoWork(project, mirror, opt, cmd, shell, cnt, config):
|
||||
setenv('REPO_REMOTE', project['remote_name'])
|
||||
setenv('REPO_LREV', project['lrev'])
|
||||
setenv('REPO_RREV', project['rrev'])
|
||||
setenv('REPO_UPSTREAM', project['upstream'])
|
||||
setenv('REPO_DEST_BRANCH', project['dest_branch'])
|
||||
setenv('REPO_I', str(cnt + 1))
|
||||
for name in project['annotations']:
|
||||
setenv("REPO__%s" % (name), project['annotations'][name])
|
||||
@ -374,8 +370,8 @@ def DoWork(project, mirror, opt, cmd, shell, cnt, config):
|
||||
for s in in_ready:
|
||||
buf = s.read().decode()
|
||||
if not buf:
|
||||
s_in.remove(s)
|
||||
s.close()
|
||||
s_in.remove(s)
|
||||
continue
|
||||
|
||||
if not opt.verbose:
|
||||
|
||||
@ -19,7 +19,6 @@ import re
|
||||
import sys
|
||||
from formatter import AbstractFormatter, DumbWriter
|
||||
|
||||
from subcmds import all_commands
|
||||
from color import Coloring
|
||||
from command import PagedCommand, MirrorSafeCommand, GitcAvailableCommand, GitcClientCommand
|
||||
import gitc_utils
|
||||
@ -43,7 +42,7 @@ Displays detailed usage information about a command.
|
||||
fmt = ' %%-%ds %%s' % maxlen
|
||||
|
||||
for name in commandNames:
|
||||
command = all_commands[name]()
|
||||
command = self.commands[name]
|
||||
try:
|
||||
summary = command.helpSummary.strip()
|
||||
except AttributeError:
|
||||
@ -53,7 +52,7 @@ Displays detailed usage information about a command.
|
||||
def _PrintAllCommands(self):
|
||||
print('usage: repo COMMAND [ARGS]')
|
||||
print('The complete list of recognized repo commands are:')
|
||||
commandNames = list(sorted(all_commands))
|
||||
commandNames = list(sorted(self.commands))
|
||||
self._PrintCommands(commandNames)
|
||||
print("See 'repo help <command>' for more information on a "
|
||||
'specific command.')
|
||||
@ -74,7 +73,7 @@ Displays detailed usage information about a command.
|
||||
return False
|
||||
|
||||
commandNames = list(sorted([name
|
||||
for name, command in all_commands.items()
|
||||
for name, command in self.commands.items()
|
||||
if command.common and gitc_supported(command)]))
|
||||
self._PrintCommands(commandNames)
|
||||
|
||||
@ -133,8 +132,8 @@ Displays detailed usage information about a command.
|
||||
out._PrintSection('Description', 'helpDescription')
|
||||
|
||||
def _PrintAllCommandHelp(self):
|
||||
for name in sorted(all_commands):
|
||||
cmd = all_commands[name]()
|
||||
for name in sorted(self.commands):
|
||||
cmd = self.commands[name]
|
||||
cmd.manifest = self.manifest
|
||||
self._PrintCommandHelp(cmd, header_prefix='[%s] ' % (name,))
|
||||
|
||||
@ -159,7 +158,7 @@ Displays detailed usage information about a command.
|
||||
name = args[0]
|
||||
|
||||
try:
|
||||
cmd = all_commands[name]()
|
||||
cmd = self.commands[name]
|
||||
except KeyError:
|
||||
print("repo: '%s' is not a repo command." % name, file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
@ -16,7 +16,7 @@
|
||||
|
||||
from command import PagedCommand
|
||||
from color import Coloring
|
||||
from git_refs import R_M, R_HEADS
|
||||
from git_refs import R_M
|
||||
|
||||
|
||||
class _Coloring(Coloring):
|
||||
@ -127,10 +127,7 @@ class Info(PagedCommand):
|
||||
if not self.opt.local:
|
||||
project.Sync_NetworkHalf(quiet=True, current_branch_only=True)
|
||||
|
||||
branch = self.manifest.manifestProject.config.GetBranch('default').merge
|
||||
if branch.startswith(R_HEADS):
|
||||
branch = branch[len(R_HEADS):]
|
||||
logTarget = R_M + branch
|
||||
logTarget = R_M + self.manifest.manifestProject.config.GetBranch("default").merge
|
||||
|
||||
bareTmp = project.bare_git._bare
|
||||
project.bare_git._bare = False
|
||||
|
||||
@ -38,7 +38,6 @@ from project import SyncBuffer
|
||||
from git_config import GitConfig
|
||||
from git_command import git_require, MIN_GIT_VERSION_SOFT, MIN_GIT_VERSION_HARD
|
||||
import platform_utils
|
||||
from wrapper import Wrapper
|
||||
|
||||
|
||||
class Init(InteractiveCommand, MirrorSafeCommand):
|
||||
@ -54,8 +53,7 @@ from the server and is installed in the .repo/ directory in the
|
||||
current working directory.
|
||||
|
||||
The optional -b argument can be used to select the manifest branch
|
||||
to checkout and use. If no branch is specified, the remote's default
|
||||
branch is used.
|
||||
to checkout and use. If no branch is specified, master is assumed.
|
||||
|
||||
The optional -m argument can be used to specify an alternate manifest
|
||||
to be used. If no manifest is specified, the manifest default.xml
|
||||
@ -156,11 +154,9 @@ to update the working directory files.
|
||||
help='restrict manifest projects to ones with a specified '
|
||||
'platform group [auto|all|none|linux|darwin|...]',
|
||||
metavar='PLATFORM')
|
||||
g.add_option('--clone-bundle', action='store_true',
|
||||
help='force use of /clone.bundle on HTTP/HTTPS (default if not --partial-clone)')
|
||||
g.add_option('--no-clone-bundle',
|
||||
dest='clone_bundle', action='store_false',
|
||||
help='disable use of /clone.bundle on HTTP/HTTPS (default if --partial-clone)')
|
||||
dest='clone_bundle', default=True, action='store_false',
|
||||
help='disable use of /clone.bundle on HTTP/HTTPS')
|
||||
g.add_option('--no-tags',
|
||||
dest='tags', default=True, action='store_false',
|
||||
help="don't fetch tags in the manifest")
|
||||
@ -170,10 +166,9 @@ to update the working directory files.
|
||||
g.add_option('--repo-url',
|
||||
dest='repo_url',
|
||||
help='repo repository location', metavar='URL')
|
||||
g.add_option('--repo-rev', metavar='REV',
|
||||
help='repo branch or revision')
|
||||
g.add_option('--repo-branch', dest='repo_rev',
|
||||
help=optparse.SUPPRESS_HELP)
|
||||
g.add_option('--repo-branch',
|
||||
dest='repo_branch',
|
||||
help='repo branch or revision', metavar='REVISION')
|
||||
g.add_option('--no-repo-verify',
|
||||
dest='repo_verify', default=True, action='store_false',
|
||||
help='do not verify repo source code')
|
||||
@ -216,27 +211,24 @@ to update the working directory files.
|
||||
|
||||
m._InitGitDir(mirror_git=mirrored_manifest_git)
|
||||
|
||||
if opt.manifest_branch:
|
||||
m.revisionExpr = opt.manifest_branch
|
||||
else:
|
||||
m.revisionExpr = 'refs/heads/master'
|
||||
else:
|
||||
if opt.manifest_branch:
|
||||
m.revisionExpr = opt.manifest_branch
|
||||
else:
|
||||
m.PreSync()
|
||||
|
||||
self._ConfigureDepth(opt)
|
||||
|
||||
# Set the remote URL before the remote branch as we might need it below.
|
||||
if opt.manifest_url:
|
||||
r = m.GetRemote(m.remote.name)
|
||||
r.url = opt.manifest_url
|
||||
r.ResetFetch()
|
||||
r.Save()
|
||||
|
||||
if opt.manifest_branch:
|
||||
m.revisionExpr = opt.manifest_branch
|
||||
else:
|
||||
if is_new:
|
||||
default_branch = m.ResolveRemoteHead()
|
||||
if default_branch is None:
|
||||
# If the remote doesn't have HEAD configured, default to master.
|
||||
default_branch = 'refs/heads/master'
|
||||
m.revisionExpr = default_branch
|
||||
else:
|
||||
m.PreSync()
|
||||
|
||||
groups = re.split(r'[,\s]+', opt.groups)
|
||||
all_platforms = ['linux', 'darwin', 'windows']
|
||||
platformize = lambda x: 'platform-' + x
|
||||
@ -309,11 +301,6 @@ to update the working directory files.
|
||||
else:
|
||||
opt.clone_filter = None
|
||||
|
||||
if opt.clone_bundle is None:
|
||||
opt.clone_bundle = False if opt.partial_clone else True
|
||||
else:
|
||||
m.config.SetString('repo.clonebundle', 'true' if opt.clone_bundle else 'false')
|
||||
|
||||
if opt.submodules:
|
||||
m.config.SetString('repo.submodules', 'true')
|
||||
|
||||
@ -492,9 +479,6 @@ to update the working directory files.
|
||||
if opt.archive and opt.mirror:
|
||||
self.OptionParser.error('--mirror and --archive cannot be used together.')
|
||||
|
||||
if args:
|
||||
self.OptionParser.error('init takes no arguments')
|
||||
|
||||
def Execute(self, opt, args):
|
||||
git_require(MIN_GIT_VERSION_HARD, fail=True)
|
||||
if not git_require(MIN_GIT_VERSION_SOFT):
|
||||
@ -506,24 +490,6 @@ to update the working directory files.
|
||||
opt.quiet = opt.output_mode is False
|
||||
opt.verbose = opt.output_mode is True
|
||||
|
||||
rp = self.manifest.repoProject
|
||||
|
||||
# Handle new --repo-url requests.
|
||||
if opt.repo_url:
|
||||
remote = rp.GetRemote('origin')
|
||||
remote.url = opt.repo_url
|
||||
remote.Save()
|
||||
|
||||
# Handle new --repo-rev requests.
|
||||
if opt.repo_rev:
|
||||
wrapper = Wrapper()
|
||||
remote_ref, rev = wrapper.check_repo_rev(
|
||||
rp.gitdir, opt.repo_rev, repo_verify=opt.repo_verify, quiet=opt.quiet)
|
||||
branch = rp.GetBranch('default')
|
||||
branch.merge = remote_ref
|
||||
rp.work_git.update_ref('refs/heads/default', rev)
|
||||
branch.Save()
|
||||
|
||||
if opt.worktree:
|
||||
# Older versions of git supported worktree, but had dangerous gc bugs.
|
||||
git_require((2, 15, 0), fail=True, msg='git gc worktree corruption')
|
||||
|
||||
@ -30,16 +30,10 @@ class Manifest(PagedCommand):
|
||||
_helpDescription = """
|
||||
|
||||
With the -o option, exports the current manifest for inspection.
|
||||
The manifest and (if present) local_manifests/ are combined
|
||||
The manifest and (if present) local_manifest.xml are combined
|
||||
together to produce a single manifest file. This file can be stored
|
||||
in a Git repository for use during future 'repo init' invocations.
|
||||
|
||||
The -r option can be used to generate a manifest file with project
|
||||
revisions set to the current commit hash. These are known as
|
||||
"revision locked manifests", as they don't follow a particular branch.
|
||||
In this case, the 'upstream' attribute is set to the ref we were on
|
||||
when the manifest was generated. The 'dest-branch' attribute is set
|
||||
to indicate the remote ref to push changes to via 'repo upload'.
|
||||
"""
|
||||
|
||||
@property
|
||||
@ -63,11 +57,6 @@ to indicate the remote ref to push changes to via 'repo upload'.
|
||||
help='If in -r mode, do not write the upstream field. '
|
||||
'Only of use if the branch names for a sha1 manifest are '
|
||||
'sensitive.')
|
||||
p.add_option('--suppress-dest-branch', dest='peg_rev_dest_branch',
|
||||
default=True, action='store_false',
|
||||
help='If in -r mode, do not write the dest-branch field. '
|
||||
'Only of use if the branch names for a sha1 manifest are '
|
||||
'sensitive.')
|
||||
p.add_option('-o', '--output-file',
|
||||
dest='output_file',
|
||||
default='-',
|
||||
@ -85,8 +74,7 @@ to indicate the remote ref to push changes to via 'repo upload'.
|
||||
fd = open(opt.output_file, 'w')
|
||||
self.manifest.Save(fd,
|
||||
peg_rev=opt.peg_rev,
|
||||
peg_rev_upstream=opt.peg_rev_upstream,
|
||||
peg_rev_dest_branch=opt.peg_rev_dest_branch)
|
||||
peg_rev_upstream=opt.peg_rev_upstream)
|
||||
fd.close()
|
||||
if opt.output_file != '-':
|
||||
print('Saved manifest to %s' % opt.output_file, file=sys.stderr)
|
||||
|
||||
@ -16,13 +16,17 @@
|
||||
|
||||
from __future__ import print_function
|
||||
|
||||
import functools
|
||||
import glob
|
||||
import multiprocessing
|
||||
import itertools
|
||||
import os
|
||||
|
||||
from command import PagedCommand
|
||||
|
||||
try:
|
||||
import threading as _threading
|
||||
except ImportError:
|
||||
import dummy_threading as _threading
|
||||
|
||||
from color import Coloring
|
||||
import platform_utils
|
||||
|
||||
@ -91,20 +95,25 @@ the following meanings:
|
||||
p.add_option('-q', '--quiet', action='store_true',
|
||||
help="only print the name of modified projects")
|
||||
|
||||
def _StatusHelper(self, quiet, project):
|
||||
def _StatusHelper(self, project, clean_counter, sem, quiet):
|
||||
"""Obtains the status for a specific project.
|
||||
|
||||
Obtains the status for a project, redirecting the output to
|
||||
the specified object.
|
||||
the specified object. It will release the semaphore
|
||||
when done.
|
||||
|
||||
Args:
|
||||
quiet: Where to output the status.
|
||||
project: Project to get status of.
|
||||
|
||||
Returns:
|
||||
The status of the project.
|
||||
clean_counter: Counter for clean projects.
|
||||
sem: Semaphore, will call release() when complete.
|
||||
output: Where to output the status.
|
||||
"""
|
||||
return project.PrintWorkTreeStatus(quiet=quiet)
|
||||
try:
|
||||
state = project.PrintWorkTreeStatus(quiet=quiet)
|
||||
if state == 'CLEAN':
|
||||
next(clean_counter)
|
||||
finally:
|
||||
sem.release()
|
||||
|
||||
def _FindOrphans(self, dirs, proj_dirs, proj_dirs_parents, outstring):
|
||||
"""find 'dirs' that are present in 'proj_dirs_parents' but not in 'proj_dirs'"""
|
||||
@ -124,18 +133,27 @@ the following meanings:
|
||||
|
||||
def Execute(self, opt, args):
|
||||
all_projects = self.GetProjects(args)
|
||||
counter = 0
|
||||
counter = itertools.count()
|
||||
|
||||
if opt.jobs == 1:
|
||||
for project in all_projects:
|
||||
state = project.PrintWorkTreeStatus(quiet=opt.quiet)
|
||||
if state == 'CLEAN':
|
||||
counter += 1
|
||||
next(counter)
|
||||
else:
|
||||
with multiprocessing.Pool(opt.jobs) as pool:
|
||||
states = pool.map(functools.partial(self._StatusHelper, opt.quiet), all_projects)
|
||||
counter += states.count('CLEAN')
|
||||
if not opt.quiet and len(all_projects) == counter:
|
||||
sem = _threading.Semaphore(opt.jobs)
|
||||
threads = []
|
||||
for project in all_projects:
|
||||
sem.acquire()
|
||||
|
||||
t = _threading.Thread(target=self._StatusHelper,
|
||||
args=(project, counter, sem, opt.quiet))
|
||||
threads.append(t)
|
||||
t.daemon = True
|
||||
t.start()
|
||||
for t in threads:
|
||||
t.join()
|
||||
if not opt.quiet and len(all_projects) == next(counter):
|
||||
print('nothing to commit (working directory clean)')
|
||||
|
||||
if opt.orphans:
|
||||
|
||||
@ -138,11 +138,11 @@ if the manifest server specified in the manifest file already includes
|
||||
credentials.
|
||||
|
||||
By default, all projects will be synced. The --fail-fast option can be used
|
||||
to halt syncing as soon as possible when the first project fails to sync.
|
||||
to halt syncing as soon as possible when the the first project fails to sync.
|
||||
|
||||
The --force-sync option can be used to overwrite existing git
|
||||
directories if they have previously been linked to a different
|
||||
object directory. WARNING: This may cause data to be lost since
|
||||
object direcotry. WARNING: This may cause data to be lost since
|
||||
refs may be removed when overwriting.
|
||||
|
||||
The --force-remove-dirty option can be used to remove previously used
|
||||
@ -247,9 +247,8 @@ later is required to fix a server side protocol bug.
|
||||
p.add_option('-m', '--manifest-name',
|
||||
dest='manifest_name',
|
||||
help='temporary manifest to use for this sync', metavar='NAME.xml')
|
||||
p.add_option('--clone-bundle', action='store_true',
|
||||
help='enable use of /clone.bundle on HTTP/HTTPS')
|
||||
p.add_option('--no-clone-bundle', dest='clone_bundle', action='store_false',
|
||||
p.add_option('--no-clone-bundle',
|
||||
dest='clone_bundle', default=True, action='store_false',
|
||||
help='disable use of /clone.bundle on HTTP/HTTPS')
|
||||
p.add_option('-u', '--manifest-server-username', action='store',
|
||||
dest='manifest_server_username',
|
||||
@ -266,9 +265,6 @@ later is required to fix a server side protocol bug.
|
||||
p.add_option('--optimized-fetch',
|
||||
dest='optimized_fetch', action='store_true',
|
||||
help='only fetch projects fixed to sha1 if revision does not exist locally')
|
||||
p.add_option('--retry-fetches',
|
||||
default=0, action='store', type='int',
|
||||
help='number of times to retry fetches on transient errors')
|
||||
p.add_option('--prune', dest='prune', action='store_true',
|
||||
help='delete refs that no longer exist on the remote')
|
||||
if show_smart:
|
||||
@ -346,7 +342,6 @@ later is required to fix a server side protocol bug.
|
||||
clone_bundle=opt.clone_bundle,
|
||||
tags=opt.tags, archive=self.manifest.IsArchive,
|
||||
optimized_fetch=opt.optimized_fetch,
|
||||
retry_fetches=opt.retry_fetches,
|
||||
prune=opt.prune,
|
||||
clone_filter=clone_filter)
|
||||
self._fetch_times.Set(project, time.time() - start)
|
||||
@ -734,12 +729,12 @@ later is required to fix a server side protocol bug.
|
||||
branch = branch[len(R_HEADS):]
|
||||
|
||||
if 'SYNC_TARGET' in os.environ:
|
||||
target = os.environ['SYNC_TARGET']
|
||||
target = os.environ('SYNC_TARGET')
|
||||
[success, manifest_str] = server.GetApprovedManifest(branch, target)
|
||||
elif ('TARGET_PRODUCT' in os.environ and
|
||||
'TARGET_BUILD_VARIANT' in os.environ):
|
||||
target = '%s-%s' % (os.environ['TARGET_PRODUCT'],
|
||||
os.environ['TARGET_BUILD_VARIANT'])
|
||||
target = '%s-%s' % (os.environ('TARGET_PRODUCT'),
|
||||
os.environ('TARGET_BUILD_VARIANT'))
|
||||
[success, manifest_str] = server.GetApprovedManifest(branch, target)
|
||||
else:
|
||||
[success, manifest_str] = server.GetApprovedManifest(branch)
|
||||
@ -782,7 +777,6 @@ later is required to fix a server side protocol bug.
|
||||
current_branch_only=opt.current_branch_only,
|
||||
tags=opt.tags,
|
||||
optimized_fetch=opt.optimized_fetch,
|
||||
retry_fetches=opt.retry_fetches,
|
||||
submodules=self.manifest.HasSubmodules,
|
||||
clone_filter=self.manifest.CloneFilter)
|
||||
finish = time.time()
|
||||
@ -837,9 +831,6 @@ later is required to fix a server side protocol bug.
|
||||
smart_sync_manifest_path = os.path.join(
|
||||
self.manifest.manifestProject.worktree, 'smart_sync_override.xml')
|
||||
|
||||
if opt.clone_bundle is None:
|
||||
opt.clone_bundle = self.manifest.CloneBundle
|
||||
|
||||
if opt.smart_sync or opt.smart_tag:
|
||||
manifest_name = self._SmartSyncSetup(opt, smart_sync_manifest_path)
|
||||
else:
|
||||
@ -854,13 +845,6 @@ later is required to fix a server side protocol bug.
|
||||
|
||||
rp = self.manifest.repoProject
|
||||
rp.PreSync()
|
||||
cb = rp.CurrentBranch
|
||||
if cb:
|
||||
base = rp.GetBranch(cb).merge
|
||||
if not base or not base.startswith('refs/heads/'):
|
||||
print('warning: repo is not tracking a remote branch, so it will not '
|
||||
'receive updates; run `repo init --repo-rev=stable` to fix.',
|
||||
file=sys.stderr)
|
||||
|
||||
mp = self.manifest.manifestProject
|
||||
mp.PreSync()
|
||||
|
||||
@ -23,8 +23,7 @@ from command import InteractiveCommand
|
||||
from editor import Editor
|
||||
from error import HookError, UploadError
|
||||
from git_command import GitCommand
|
||||
from git_refs import R_HEADS
|
||||
from hooks import RepoHook
|
||||
from project import RepoHook
|
||||
|
||||
from pyversion import is_python3
|
||||
if not is_python3():
|
||||
@ -135,18 +134,7 @@ review.URL.uploadhashtags:
|
||||
|
||||
To add hashtags whenever uploading a commit, you can set a per-project
|
||||
or global Git option to do so. The value of review.URL.uploadhashtags
|
||||
will be used as comma delimited hashtags like the --hashtag option.
|
||||
|
||||
review.URL.uploadlabels:
|
||||
|
||||
To add labels whenever uploading a commit, you can set a per-project
|
||||
or global Git option to do so. The value of review.URL.uploadlabels
|
||||
will be used as comma delimited labels like the --label option.
|
||||
|
||||
review.URL.uploadnotify:
|
||||
|
||||
Control e-mail notifications when uploading.
|
||||
https://gerrit-review.googlesource.com/Documentation/user-upload.html#notify
|
||||
will be used as comma delimited hashtags like the --hashtags option.
|
||||
|
||||
# References
|
||||
|
||||
@ -164,9 +152,6 @@ Gerrit Code Review: https://www.gerritcodereview.com/
|
||||
p.add_option('--hashtag-branch', '--htb',
|
||||
action='store_true',
|
||||
help='Add local branch name as a hashtag.')
|
||||
p.add_option('-l', '--label',
|
||||
dest='labels', action='append', default=[],
|
||||
help='Add a label when uploading.')
|
||||
p.add_option('--re', '--reviewers',
|
||||
type='string', action='append', dest='reviewers',
|
||||
help='Request reviews from these people.')
|
||||
@ -179,6 +164,9 @@ Gerrit Code Review: https://www.gerritcodereview.com/
|
||||
p.add_option('--cbr', '--current-branch',
|
||||
dest='current_branch', action='store_true',
|
||||
help='Upload current git branch.')
|
||||
p.add_option('-d', '--draft',
|
||||
action='store_true', dest='draft', default=False,
|
||||
help='If specified, upload as a draft.')
|
||||
p.add_option('--ne', '--no-emails',
|
||||
action='store_false', dest='notify', default=True,
|
||||
help='If specified, do not send emails on upload.')
|
||||
@ -250,7 +238,7 @@ Gerrit Code Review: https://www.gerritcodereview.com/
|
||||
|
||||
destination = opt.dest_branch or project.dest_branch or project.revisionExpr
|
||||
print('Upload project %s/ to remote branch %s%s:' %
|
||||
(project.relpath, destination, ' (private)' if opt.private else ''))
|
||||
(project.relpath, destination, ' (draft)' if opt.draft else ''))
|
||||
print(' branch %s (%2d commit%s, %s):' % (
|
||||
name,
|
||||
len(commit_list),
|
||||
@ -422,51 +410,28 @@ Gerrit Code Review: https://www.gerritcodereview.com/
|
||||
key = 'review.%s.uploadtopic' % branch.project.remote.review
|
||||
opt.auto_topic = branch.project.config.GetBoolean(key)
|
||||
|
||||
def _ExpandCommaList(value):
|
||||
"""Split |value| up into comma delimited entries."""
|
||||
# Check if hashtags should be included.
|
||||
def _ExpandHashtag(value):
|
||||
"""Split |value| up into comma delimited tags."""
|
||||
if not value:
|
||||
return
|
||||
for ret in value.split(','):
|
||||
ret = ret.strip()
|
||||
if ret:
|
||||
yield ret
|
||||
|
||||
# Check if hashtags should be included.
|
||||
for tag in value.split(','):
|
||||
tag = tag.strip()
|
||||
if tag:
|
||||
yield tag
|
||||
key = 'review.%s.uploadhashtags' % branch.project.remote.review
|
||||
hashtags = set(_ExpandCommaList(branch.project.config.GetString(key)))
|
||||
hashtags = set(_ExpandHashtag(branch.project.config.GetString(key)))
|
||||
for tag in opt.hashtags:
|
||||
hashtags.update(_ExpandCommaList(tag))
|
||||
hashtags.update(_ExpandHashtag(tag))
|
||||
if opt.hashtag_branch:
|
||||
hashtags.add(branch.name)
|
||||
|
||||
# Check if labels should be included.
|
||||
key = 'review.%s.uploadlabels' % branch.project.remote.review
|
||||
labels = set(_ExpandCommaList(branch.project.config.GetString(key)))
|
||||
for label in opt.labels:
|
||||
labels.update(_ExpandCommaList(label))
|
||||
# Basic sanity check on label syntax.
|
||||
for label in labels:
|
||||
if not re.match(r'^.+[+-][0-9]+$', label):
|
||||
print('repo: error: invalid label syntax "%s": labels use forms '
|
||||
'like CodeReview+1 or Verified-1' % (label,), file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
# Handle e-mail notifications.
|
||||
if opt.notify is False:
|
||||
notify = 'NONE'
|
||||
else:
|
||||
key = 'review.%s.uploadnotify' % branch.project.remote.review
|
||||
notify = branch.project.config.GetString(key)
|
||||
|
||||
destination = opt.dest_branch or branch.project.dest_branch
|
||||
|
||||
# Make sure our local branch is not setup to track a different remote branch
|
||||
merge_branch = self._GetMergeBranch(branch.project)
|
||||
if destination:
|
||||
full_dest = destination
|
||||
if not full_dest.startswith(R_HEADS):
|
||||
full_dest = R_HEADS + full_dest
|
||||
|
||||
full_dest = 'refs/heads/%s' % destination
|
||||
if not opt.dest_branch and merge_branch and merge_branch != full_dest:
|
||||
print('merge branch %s does not match destination branch %s'
|
||||
% (merge_branch, full_dest))
|
||||
@ -480,9 +445,9 @@ Gerrit Code Review: https://www.gerritcodereview.com/
|
||||
dryrun=opt.dryrun,
|
||||
auto_topic=opt.auto_topic,
|
||||
hashtags=hashtags,
|
||||
labels=labels,
|
||||
draft=opt.draft,
|
||||
private=opt.private,
|
||||
notify=notify,
|
||||
notify=None if opt.notify else 'NONE',
|
||||
wip=opt.wip,
|
||||
dest_branch=destination,
|
||||
validate_certs=opt.validate_certs,
|
||||
@ -596,7 +561,7 @@ Gerrit Code Review: https://www.gerritcodereview.com/
|
||||
print('\nWARNING: pre-upload hooks failed, but uploading anyways.',
|
||||
file=sys.stderr)
|
||||
else:
|
||||
return 1
|
||||
return
|
||||
|
||||
if opt.reviewers:
|
||||
reviewers = _SplitEmails(opt.reviewers)
|
||||
|
||||
@ -15,10 +15,7 @@
|
||||
# limitations under the License.
|
||||
|
||||
from __future__ import print_function
|
||||
|
||||
import platform
|
||||
import sys
|
||||
|
||||
from command import Command, MirrorSafeCommand
|
||||
from git_command import git, RepoSourceVersion, user_agent
|
||||
from git_refs import HEAD
|
||||
@ -43,11 +40,10 @@ class Version(Command, MirrorSafeCommand):
|
||||
rp_ver = rp.bare_git.describe(HEAD)
|
||||
print('repo version %s' % rp_ver)
|
||||
print(' (from %s)' % rem.url)
|
||||
print(' (%s)' % rp.bare_git.log('-1', '--format=%cD', HEAD))
|
||||
|
||||
if self.wrapper_path is not None:
|
||||
print('repo launcher version %s' % self.wrapper_version)
|
||||
print(' (from %s)' % self.wrapper_path)
|
||||
if Version.wrapper_path is not None:
|
||||
print('repo launcher version %s' % Version.wrapper_version)
|
||||
print(' (from %s)' % Version.wrapper_path)
|
||||
|
||||
if src_ver != rp_ver:
|
||||
print(' (currently at %s)' % src_ver)
|
||||
@ -56,11 +52,3 @@ class Version(Command, MirrorSafeCommand):
|
||||
print('git %s' % git.version_tuple().full)
|
||||
print('git User-Agent %s' % user_agent.git)
|
||||
print('Python %s' % sys.version)
|
||||
uname = platform.uname()
|
||||
if sys.version_info.major < 3:
|
||||
# Python 3 returns a named tuple, but Python 2 is simpler.
|
||||
print(uname)
|
||||
else:
|
||||
print('OS %s %s (%s)' % (uname.system, uname.release, uname.version))
|
||||
print('CPU %s (%s)' %
|
||||
(uname.machine, uname.processor if uname.processor else 'unknown'))
|
||||
|
||||
@ -30,33 +30,6 @@ import git_command
|
||||
import wrapper
|
||||
|
||||
|
||||
class SSHUnitTest(unittest.TestCase):
|
||||
"""Tests the ssh functions."""
|
||||
|
||||
def test_ssh_version(self):
|
||||
"""Check ssh_version() handling."""
|
||||
ver = git_command._parse_ssh_version('Unknown\n')
|
||||
self.assertEqual(ver, ())
|
||||
ver = git_command._parse_ssh_version('OpenSSH_1.0\n')
|
||||
self.assertEqual(ver, (1, 0))
|
||||
ver = git_command._parse_ssh_version('OpenSSH_6.6.1p1 Ubuntu-2ubuntu2.13, OpenSSL 1.0.1f 6 Jan 2014\n')
|
||||
self.assertEqual(ver, (6, 6, 1))
|
||||
ver = git_command._parse_ssh_version('OpenSSH_7.6p1 Ubuntu-4ubuntu0.3, OpenSSL 1.0.2n 7 Dec 2017\n')
|
||||
self.assertEqual(ver, (7, 6))
|
||||
|
||||
def test_ssh_sock(self):
|
||||
"""Check ssh_sock() function."""
|
||||
with mock.patch('tempfile.mkdtemp', return_value='/tmp/foo'):
|
||||
# old ssh version uses port
|
||||
with mock.patch('git_command.ssh_version', return_value=(6, 6)):
|
||||
self.assertTrue(git_command.ssh_sock().endswith('%p'))
|
||||
git_command._ssh_sock_path = None
|
||||
# new ssh version uses hash
|
||||
with mock.patch('git_command.ssh_version', return_value=(6, 7)):
|
||||
self.assertTrue(git_command.ssh_sock().endswith('%C'))
|
||||
git_command._ssh_sock_path = None
|
||||
|
||||
|
||||
class GitCallUnitTest(unittest.TestCase):
|
||||
"""Tests the _GitCall class (via git_command.git)."""
|
||||
|
||||
|
||||
@ -1,60 +0,0 @@
|
||||
# -*- coding:utf-8 -*-
|
||||
#
|
||||
# Copyright (C) 2019 The Android Open Source Project
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
"""Unittests for the hooks.py module."""
|
||||
|
||||
from __future__ import print_function
|
||||
|
||||
import hooks
|
||||
import unittest
|
||||
|
||||
class RepoHookShebang(unittest.TestCase):
|
||||
"""Check shebang parsing in RepoHook."""
|
||||
|
||||
def test_no_shebang(self):
|
||||
"""Lines w/out shebangs should be rejected."""
|
||||
DATA = (
|
||||
'',
|
||||
'# -*- coding:utf-8 -*-\n',
|
||||
'#\n# foo\n',
|
||||
'# Bad shebang in script\n#!/foo\n'
|
||||
)
|
||||
for data in DATA:
|
||||
self.assertIsNone(hooks.RepoHook._ExtractInterpFromShebang(data))
|
||||
|
||||
def test_direct_interp(self):
|
||||
"""Lines whose shebang points directly to the interpreter."""
|
||||
DATA = (
|
||||
('#!/foo', '/foo'),
|
||||
('#! /foo', '/foo'),
|
||||
('#!/bin/foo ', '/bin/foo'),
|
||||
('#! /usr/foo ', '/usr/foo'),
|
||||
('#! /usr/foo -args', '/usr/foo'),
|
||||
)
|
||||
for shebang, interp in DATA:
|
||||
self.assertEqual(hooks.RepoHook._ExtractInterpFromShebang(shebang),
|
||||
interp)
|
||||
|
||||
def test_env_interp(self):
|
||||
"""Lines whose shebang launches through `env`."""
|
||||
DATA = (
|
||||
('#!/usr/bin/env foo', 'foo'),
|
||||
('#!/bin/env foo', 'foo'),
|
||||
('#! /bin/env /bin/foo ', '/bin/foo'),
|
||||
)
|
||||
for shebang, interp in DATA:
|
||||
self.assertEqual(hooks.RepoHook._ExtractInterpFromShebang(shebang),
|
||||
interp)
|
||||
@ -20,7 +20,6 @@ from __future__ import print_function
|
||||
|
||||
import os
|
||||
import unittest
|
||||
import xml.dom.minidom
|
||||
|
||||
import error
|
||||
import manifest_xml
|
||||
@ -90,59 +89,3 @@ class ManifestValidateFilePaths(unittest.TestCase):
|
||||
error.ManifestInvalidPathError, self.check_both, path, 'a')
|
||||
self.assertRaises(
|
||||
error.ManifestInvalidPathError, self.check_both, 'a', path)
|
||||
|
||||
|
||||
class ValueTests(unittest.TestCase):
|
||||
"""Check utility parsing code."""
|
||||
|
||||
def _get_node(self, text):
|
||||
return xml.dom.minidom.parseString(text).firstChild
|
||||
|
||||
def test_bool_default(self):
|
||||
"""Check XmlBool default handling."""
|
||||
node = self._get_node('<node/>')
|
||||
self.assertIsNone(manifest_xml.XmlBool(node, 'a'))
|
||||
self.assertIsNone(manifest_xml.XmlBool(node, 'a', None))
|
||||
self.assertEqual(123, manifest_xml.XmlBool(node, 'a', 123))
|
||||
|
||||
node = self._get_node('<node a=""/>')
|
||||
self.assertIsNone(manifest_xml.XmlBool(node, 'a'))
|
||||
|
||||
def test_bool_invalid(self):
|
||||
"""Check XmlBool invalid handling."""
|
||||
node = self._get_node('<node a="moo"/>')
|
||||
self.assertEqual(123, manifest_xml.XmlBool(node, 'a', 123))
|
||||
|
||||
def test_bool_true(self):
|
||||
"""Check XmlBool true values."""
|
||||
for value in ('yes', 'true', '1'):
|
||||
node = self._get_node('<node a="%s"/>' % (value,))
|
||||
self.assertTrue(manifest_xml.XmlBool(node, 'a'))
|
||||
|
||||
def test_bool_false(self):
|
||||
"""Check XmlBool false values."""
|
||||
for value in ('no', 'false', '0'):
|
||||
node = self._get_node('<node a="%s"/>' % (value,))
|
||||
self.assertFalse(manifest_xml.XmlBool(node, 'a'))
|
||||
|
||||
def test_int_default(self):
|
||||
"""Check XmlInt default handling."""
|
||||
node = self._get_node('<node/>')
|
||||
self.assertIsNone(manifest_xml.XmlInt(node, 'a'))
|
||||
self.assertIsNone(manifest_xml.XmlInt(node, 'a', None))
|
||||
self.assertEqual(123, manifest_xml.XmlInt(node, 'a', 123))
|
||||
|
||||
node = self._get_node('<node a=""/>')
|
||||
self.assertIsNone(manifest_xml.XmlInt(node, 'a'))
|
||||
|
||||
def test_int_good(self):
|
||||
"""Check XmlInt numeric handling."""
|
||||
for value in (-1, 0, 1, 50000):
|
||||
node = self._get_node('<node a="%s"/>' % (value,))
|
||||
self.assertEqual(value, manifest_xml.XmlInt(node, 'a'))
|
||||
|
||||
def test_int_invalid(self):
|
||||
"""Check XmlInt invalid handling."""
|
||||
with self.assertRaises(error.ManifestParseError):
|
||||
node = self._get_node('<node a="xx"/>')
|
||||
manifest_xml.XmlInt(node, 'a')
|
||||
|
||||
@ -44,6 +44,45 @@ def TempGitTree():
|
||||
platform_utils.rmtree(tempdir)
|
||||
|
||||
|
||||
class RepoHookShebang(unittest.TestCase):
|
||||
"""Check shebang parsing in RepoHook."""
|
||||
|
||||
def test_no_shebang(self):
|
||||
"""Lines w/out shebangs should be rejected."""
|
||||
DATA = (
|
||||
'',
|
||||
'# -*- coding:utf-8 -*-\n',
|
||||
'#\n# foo\n',
|
||||
'# Bad shebang in script\n#!/foo\n'
|
||||
)
|
||||
for data in DATA:
|
||||
self.assertIsNone(project.RepoHook._ExtractInterpFromShebang(data))
|
||||
|
||||
def test_direct_interp(self):
|
||||
"""Lines whose shebang points directly to the interpreter."""
|
||||
DATA = (
|
||||
('#!/foo', '/foo'),
|
||||
('#! /foo', '/foo'),
|
||||
('#!/bin/foo ', '/bin/foo'),
|
||||
('#! /usr/foo ', '/usr/foo'),
|
||||
('#! /usr/foo -args', '/usr/foo'),
|
||||
)
|
||||
for shebang, interp in DATA:
|
||||
self.assertEqual(project.RepoHook._ExtractInterpFromShebang(shebang),
|
||||
interp)
|
||||
|
||||
def test_env_interp(self):
|
||||
"""Lines whose shebang launches through `env`."""
|
||||
DATA = (
|
||||
('#!/usr/bin/env foo', 'foo'),
|
||||
('#!/bin/env foo', 'foo'),
|
||||
('#! /bin/env /bin/foo ', '/bin/foo'),
|
||||
)
|
||||
for shebang, interp in DATA:
|
||||
self.assertEqual(project.RepoHook._ExtractInterpFromShebang(shebang),
|
||||
interp)
|
||||
|
||||
|
||||
class FakeProject(object):
|
||||
"""A fake for Project for basic functionality."""
|
||||
|
||||
|
||||
@ -1,43 +0,0 @@
|
||||
# Copyright (C) 2020 The Android Open Source Project
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
"""Unittests for the subcmds module (mostly __init__.py than subcommands)."""
|
||||
|
||||
import unittest
|
||||
|
||||
import subcmds
|
||||
|
||||
|
||||
class AllCommands(unittest.TestCase):
|
||||
"""Check registered all_commands."""
|
||||
|
||||
def test_required_basic(self):
|
||||
"""Basic checking of registered commands."""
|
||||
# NB: We don't test all subcommands as we want to avoid "change detection"
|
||||
# tests, so we just look for the most common/important ones here that are
|
||||
# unlikely to ever change.
|
||||
for cmd in {'cherry-pick', 'help', 'init', 'start', 'sync', 'upload'}:
|
||||
self.assertIn(cmd, subcmds.all_commands)
|
||||
|
||||
def test_naming(self):
|
||||
"""Verify we don't add things that we shouldn't."""
|
||||
for cmd in subcmds.all_commands:
|
||||
# Reject filename suffixes like "help.py".
|
||||
self.assertNotIn('.', cmd)
|
||||
|
||||
# Make sure all '_' were converted to '-'.
|
||||
self.assertNotIn('_', cmd)
|
||||
|
||||
# Reject internal python paths like "__init__".
|
||||
self.assertFalse(cmd.startswith('__'))
|
||||
@ -1,49 +0,0 @@
|
||||
# Copyright (C) 2020 The Android Open Source Project
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
"""Unittests for the subcmds/init.py module."""
|
||||
|
||||
import unittest
|
||||
|
||||
from subcmds import init
|
||||
|
||||
|
||||
class InitCommand(unittest.TestCase):
|
||||
"""Check registered all_commands."""
|
||||
|
||||
def setUp(self):
|
||||
self.cmd = init.Init()
|
||||
|
||||
def test_cli_parser_good(self):
|
||||
"""Check valid command line options."""
|
||||
ARGV = (
|
||||
[],
|
||||
)
|
||||
for argv in ARGV:
|
||||
opts, args = self.cmd.OptionParser.parse_args(argv)
|
||||
self.cmd.ValidateOptions(opts, args)
|
||||
|
||||
def test_cli_parser_bad(self):
|
||||
"""Check invalid command line options."""
|
||||
ARGV = (
|
||||
# Too many arguments.
|
||||
['asdf'],
|
||||
|
||||
# Conflicting options.
|
||||
['--mirror', '--archive'],
|
||||
)
|
||||
for argv in ARGV:
|
||||
opts, args = self.cmd.OptionParser.parse_args(argv)
|
||||
with self.assertRaises(SystemExit):
|
||||
self.cmd.ValidateOptions(opts, args)
|
||||
@ -18,14 +18,10 @@
|
||||
|
||||
from __future__ import print_function
|
||||
|
||||
import contextlib
|
||||
import os
|
||||
import re
|
||||
import shutil
|
||||
import tempfile
|
||||
import unittest
|
||||
|
||||
import platform_utils
|
||||
from pyversion import is_python3
|
||||
import wrapper
|
||||
|
||||
@ -38,18 +34,6 @@ else:
|
||||
from StringIO import StringIO
|
||||
|
||||
|
||||
@contextlib.contextmanager
|
||||
def TemporaryDirectory():
|
||||
"""Create a new empty git checkout for testing."""
|
||||
# TODO(vapier): Convert this to tempfile.TemporaryDirectory once we drop
|
||||
# Python 2 support entirely.
|
||||
try:
|
||||
tempdir = tempfile.mkdtemp(prefix='repo-tests')
|
||||
yield tempdir
|
||||
finally:
|
||||
platform_utils.rmtree(tempdir)
|
||||
|
||||
|
||||
def fixture(*paths):
|
||||
"""Return a path relative to tests/fixtures.
|
||||
"""
|
||||
@ -169,317 +153,5 @@ class SetGitTrace2ParentSid(RepoWrapperTestCase):
|
||||
self.assertRegex(value, self.VALID_FORMAT)
|
||||
|
||||
|
||||
class RunCommand(RepoWrapperTestCase):
|
||||
"""Check run_command behavior."""
|
||||
|
||||
def test_capture(self):
|
||||
"""Check capture_output handling."""
|
||||
ret = self.wrapper.run_command(['echo', 'hi'], capture_output=True)
|
||||
self.assertEqual(ret.stdout, 'hi\n')
|
||||
|
||||
def test_check(self):
|
||||
"""Check check handling."""
|
||||
self.wrapper.run_command(['true'], check=False)
|
||||
self.wrapper.run_command(['true'], check=True)
|
||||
self.wrapper.run_command(['false'], check=False)
|
||||
with self.assertRaises(self.wrapper.RunError):
|
||||
self.wrapper.run_command(['false'], check=True)
|
||||
|
||||
|
||||
class RunGit(RepoWrapperTestCase):
|
||||
"""Check run_git behavior."""
|
||||
|
||||
def test_capture(self):
|
||||
"""Check capture_output handling."""
|
||||
ret = self.wrapper.run_git('--version')
|
||||
self.assertIn('git', ret.stdout)
|
||||
|
||||
def test_check(self):
|
||||
"""Check check handling."""
|
||||
with self.assertRaises(self.wrapper.CloneFailure):
|
||||
self.wrapper.run_git('--version-asdfasdf')
|
||||
self.wrapper.run_git('--version-asdfasdf', check=False)
|
||||
|
||||
|
||||
class ParseGitVersion(RepoWrapperTestCase):
|
||||
"""Check ParseGitVersion behavior."""
|
||||
|
||||
def test_autoload(self):
|
||||
"""Check we can load the version from the live git."""
|
||||
ret = self.wrapper.ParseGitVersion()
|
||||
self.assertIsNotNone(ret)
|
||||
|
||||
def test_bad_ver(self):
|
||||
"""Check handling of bad git versions."""
|
||||
ret = self.wrapper.ParseGitVersion(ver_str='asdf')
|
||||
self.assertIsNone(ret)
|
||||
|
||||
def test_normal_ver(self):
|
||||
"""Check handling of normal git versions."""
|
||||
ret = self.wrapper.ParseGitVersion(ver_str='git version 2.25.1')
|
||||
self.assertEqual(2, ret.major)
|
||||
self.assertEqual(25, ret.minor)
|
||||
self.assertEqual(1, ret.micro)
|
||||
self.assertEqual('2.25.1', ret.full)
|
||||
|
||||
def test_extended_ver(self):
|
||||
"""Check handling of extended distro git versions."""
|
||||
ret = self.wrapper.ParseGitVersion(
|
||||
ver_str='git version 1.30.50.696.g5e7596f4ac-goog')
|
||||
self.assertEqual(1, ret.major)
|
||||
self.assertEqual(30, ret.minor)
|
||||
self.assertEqual(50, ret.micro)
|
||||
self.assertEqual('1.30.50.696.g5e7596f4ac-goog', ret.full)
|
||||
|
||||
|
||||
class CheckGitVersion(RepoWrapperTestCase):
|
||||
"""Check _CheckGitVersion behavior."""
|
||||
|
||||
def test_unknown(self):
|
||||
"""Unknown versions should abort."""
|
||||
with mock.patch.object(self.wrapper, 'ParseGitVersion', return_value=None):
|
||||
with self.assertRaises(self.wrapper.CloneFailure):
|
||||
self.wrapper._CheckGitVersion()
|
||||
|
||||
def test_old(self):
|
||||
"""Old versions should abort."""
|
||||
with mock.patch.object(
|
||||
self.wrapper, 'ParseGitVersion',
|
||||
return_value=self.wrapper.GitVersion(1, 0, 0, '1.0.0')):
|
||||
with self.assertRaises(self.wrapper.CloneFailure):
|
||||
self.wrapper._CheckGitVersion()
|
||||
|
||||
def test_new(self):
|
||||
"""Newer versions should run fine."""
|
||||
with mock.patch.object(
|
||||
self.wrapper, 'ParseGitVersion',
|
||||
return_value=self.wrapper.GitVersion(100, 0, 0, '100.0.0')):
|
||||
self.wrapper._CheckGitVersion()
|
||||
|
||||
|
||||
class NeedSetupGnuPG(RepoWrapperTestCase):
|
||||
"""Check NeedSetupGnuPG behavior."""
|
||||
|
||||
def test_missing_dir(self):
|
||||
"""The ~/.repoconfig tree doesn't exist yet."""
|
||||
with TemporaryDirectory() as tempdir:
|
||||
self.wrapper.home_dot_repo = os.path.join(tempdir, 'foo')
|
||||
self.assertTrue(self.wrapper.NeedSetupGnuPG())
|
||||
|
||||
def test_missing_keyring(self):
|
||||
"""The keyring-version file doesn't exist yet."""
|
||||
with TemporaryDirectory() as tempdir:
|
||||
self.wrapper.home_dot_repo = tempdir
|
||||
self.assertTrue(self.wrapper.NeedSetupGnuPG())
|
||||
|
||||
def test_empty_keyring(self):
|
||||
"""The keyring-version file exists, but is empty."""
|
||||
with TemporaryDirectory() as tempdir:
|
||||
self.wrapper.home_dot_repo = tempdir
|
||||
with open(os.path.join(tempdir, 'keyring-version'), 'w'):
|
||||
pass
|
||||
self.assertTrue(self.wrapper.NeedSetupGnuPG())
|
||||
|
||||
def test_old_keyring(self):
|
||||
"""The keyring-version file exists, but it's old."""
|
||||
with TemporaryDirectory() as tempdir:
|
||||
self.wrapper.home_dot_repo = tempdir
|
||||
with open(os.path.join(tempdir, 'keyring-version'), 'w') as fp:
|
||||
fp.write('1.0\n')
|
||||
self.assertTrue(self.wrapper.NeedSetupGnuPG())
|
||||
|
||||
def test_new_keyring(self):
|
||||
"""The keyring-version file exists, and is up-to-date."""
|
||||
with TemporaryDirectory() as tempdir:
|
||||
self.wrapper.home_dot_repo = tempdir
|
||||
with open(os.path.join(tempdir, 'keyring-version'), 'w') as fp:
|
||||
fp.write('1000.0\n')
|
||||
self.assertFalse(self.wrapper.NeedSetupGnuPG())
|
||||
|
||||
|
||||
class SetupGnuPG(RepoWrapperTestCase):
|
||||
"""Check SetupGnuPG behavior."""
|
||||
|
||||
def test_full(self):
|
||||
"""Make sure it works completely."""
|
||||
with TemporaryDirectory() as tempdir:
|
||||
self.wrapper.home_dot_repo = tempdir
|
||||
self.wrapper.gpg_dir = os.path.join(self.wrapper.home_dot_repo, 'gnupg')
|
||||
self.assertTrue(self.wrapper.SetupGnuPG(True))
|
||||
with open(os.path.join(tempdir, 'keyring-version'), 'r') as fp:
|
||||
data = fp.read()
|
||||
self.assertEqual('.'.join(str(x) for x in self.wrapper.KEYRING_VERSION),
|
||||
data.strip())
|
||||
|
||||
|
||||
class VerifyRev(RepoWrapperTestCase):
|
||||
"""Check verify_rev behavior."""
|
||||
|
||||
def test_verify_passes(self):
|
||||
"""Check when we have a valid signed tag."""
|
||||
desc_result = self.wrapper.RunResult(0, 'v1.0\n', '')
|
||||
gpg_result = self.wrapper.RunResult(0, '', '')
|
||||
with mock.patch.object(self.wrapper, 'run_git',
|
||||
side_effect=(desc_result, gpg_result)):
|
||||
ret = self.wrapper.verify_rev('/', 'refs/heads/stable', '1234', True)
|
||||
self.assertEqual('v1.0^0', ret)
|
||||
|
||||
def test_unsigned_commit(self):
|
||||
"""Check we fall back to signed tag when we have an unsigned commit."""
|
||||
desc_result = self.wrapper.RunResult(0, 'v1.0-10-g1234\n', '')
|
||||
gpg_result = self.wrapper.RunResult(0, '', '')
|
||||
with mock.patch.object(self.wrapper, 'run_git',
|
||||
side_effect=(desc_result, gpg_result)):
|
||||
ret = self.wrapper.verify_rev('/', 'refs/heads/stable', '1234', True)
|
||||
self.assertEqual('v1.0^0', ret)
|
||||
|
||||
def test_verify_fails(self):
|
||||
"""Check we fall back to signed tag when we have an unsigned commit."""
|
||||
desc_result = self.wrapper.RunResult(0, 'v1.0-10-g1234\n', '')
|
||||
gpg_result = Exception
|
||||
with mock.patch.object(self.wrapper, 'run_git',
|
||||
side_effect=(desc_result, gpg_result)):
|
||||
with self.assertRaises(Exception):
|
||||
self.wrapper.verify_rev('/', 'refs/heads/stable', '1234', True)
|
||||
|
||||
|
||||
class GitCheckoutTestCase(RepoWrapperTestCase):
|
||||
"""Tests that use a real/small git checkout."""
|
||||
|
||||
GIT_DIR = None
|
||||
REV_LIST = None
|
||||
|
||||
@classmethod
|
||||
def setUpClass(cls):
|
||||
# Create a repo to operate on, but do it once per-class.
|
||||
cls.GIT_DIR = tempfile.mkdtemp(prefix='repo-rev-tests')
|
||||
run_git = wrapper.Wrapper().run_git
|
||||
|
||||
remote = os.path.join(cls.GIT_DIR, 'remote')
|
||||
os.mkdir(remote)
|
||||
run_git('init', cwd=remote)
|
||||
run_git('commit', '--allow-empty', '-minit', cwd=remote)
|
||||
run_git('branch', 'stable', cwd=remote)
|
||||
run_git('tag', 'v1.0', cwd=remote)
|
||||
run_git('commit', '--allow-empty', '-m2nd commit', cwd=remote)
|
||||
cls.REV_LIST = run_git('rev-list', 'HEAD', cwd=remote).stdout.splitlines()
|
||||
|
||||
run_git('init', cwd=cls.GIT_DIR)
|
||||
run_git('fetch', remote, '+refs/heads/*:refs/remotes/origin/*', cwd=cls.GIT_DIR)
|
||||
|
||||
@classmethod
|
||||
def tearDownClass(cls):
|
||||
if not cls.GIT_DIR:
|
||||
return
|
||||
|
||||
shutil.rmtree(cls.GIT_DIR)
|
||||
|
||||
|
||||
class ResolveRepoRev(GitCheckoutTestCase):
|
||||
"""Check resolve_repo_rev behavior."""
|
||||
|
||||
def test_explicit_branch(self):
|
||||
"""Check refs/heads/branch argument."""
|
||||
rrev, lrev = self.wrapper.resolve_repo_rev(self.GIT_DIR, 'refs/heads/stable')
|
||||
self.assertEqual('refs/heads/stable', rrev)
|
||||
self.assertEqual(self.REV_LIST[1], lrev)
|
||||
|
||||
with self.assertRaises(wrapper.CloneFailure):
|
||||
self.wrapper.resolve_repo_rev(self.GIT_DIR, 'refs/heads/unknown')
|
||||
|
||||
def test_explicit_tag(self):
|
||||
"""Check refs/tags/tag argument."""
|
||||
rrev, lrev = self.wrapper.resolve_repo_rev(self.GIT_DIR, 'refs/tags/v1.0')
|
||||
self.assertEqual('refs/tags/v1.0', rrev)
|
||||
self.assertEqual(self.REV_LIST[1], lrev)
|
||||
|
||||
with self.assertRaises(wrapper.CloneFailure):
|
||||
self.wrapper.resolve_repo_rev(self.GIT_DIR, 'refs/tags/unknown')
|
||||
|
||||
def test_branch_name(self):
|
||||
"""Check branch argument."""
|
||||
rrev, lrev = self.wrapper.resolve_repo_rev(self.GIT_DIR, 'stable')
|
||||
self.assertEqual('refs/heads/stable', rrev)
|
||||
self.assertEqual(self.REV_LIST[1], lrev)
|
||||
|
||||
rrev, lrev = self.wrapper.resolve_repo_rev(self.GIT_DIR, 'master')
|
||||
self.assertEqual('refs/heads/master', rrev)
|
||||
self.assertEqual(self.REV_LIST[0], lrev)
|
||||
|
||||
def test_tag_name(self):
|
||||
"""Check tag argument."""
|
||||
rrev, lrev = self.wrapper.resolve_repo_rev(self.GIT_DIR, 'v1.0')
|
||||
self.assertEqual('refs/tags/v1.0', rrev)
|
||||
self.assertEqual(self.REV_LIST[1], lrev)
|
||||
|
||||
def test_full_commit(self):
|
||||
"""Check specific commit argument."""
|
||||
commit = self.REV_LIST[0]
|
||||
rrev, lrev = self.wrapper.resolve_repo_rev(self.GIT_DIR, commit)
|
||||
self.assertEqual(commit, rrev)
|
||||
self.assertEqual(commit, lrev)
|
||||
|
||||
def test_partial_commit(self):
|
||||
"""Check specific (partial) commit argument."""
|
||||
commit = self.REV_LIST[0][0:20]
|
||||
rrev, lrev = self.wrapper.resolve_repo_rev(self.GIT_DIR, commit)
|
||||
self.assertEqual(self.REV_LIST[0], rrev)
|
||||
self.assertEqual(self.REV_LIST[0], lrev)
|
||||
|
||||
def test_unknown(self):
|
||||
"""Check unknown ref/commit argument."""
|
||||
with self.assertRaises(wrapper.CloneFailure):
|
||||
self.wrapper.resolve_repo_rev(self.GIT_DIR, 'boooooooya')
|
||||
|
||||
|
||||
class CheckRepoVerify(RepoWrapperTestCase):
|
||||
"""Check check_repo_verify behavior."""
|
||||
|
||||
def test_no_verify(self):
|
||||
"""Always fail with --no-repo-verify."""
|
||||
self.assertFalse(self.wrapper.check_repo_verify(False))
|
||||
|
||||
def test_gpg_initialized(self):
|
||||
"""Should pass if gpg is setup already."""
|
||||
with mock.patch.object(self.wrapper, 'NeedSetupGnuPG', return_value=False):
|
||||
self.assertTrue(self.wrapper.check_repo_verify(True))
|
||||
|
||||
def test_need_gpg_setup(self):
|
||||
"""Should pass/fail based on gpg setup."""
|
||||
with mock.patch.object(self.wrapper, 'NeedSetupGnuPG', return_value=True):
|
||||
with mock.patch.object(self.wrapper, 'SetupGnuPG') as m:
|
||||
m.return_value = True
|
||||
self.assertTrue(self.wrapper.check_repo_verify(True))
|
||||
|
||||
m.return_value = False
|
||||
self.assertFalse(self.wrapper.check_repo_verify(True))
|
||||
|
||||
|
||||
class CheckRepoRev(GitCheckoutTestCase):
|
||||
"""Check check_repo_rev behavior."""
|
||||
|
||||
def test_verify_works(self):
|
||||
"""Should pass when verification passes."""
|
||||
with mock.patch.object(self.wrapper, 'check_repo_verify', return_value=True):
|
||||
with mock.patch.object(self.wrapper, 'verify_rev', return_value='12345'):
|
||||
rrev, lrev = self.wrapper.check_repo_rev(self.GIT_DIR, 'stable')
|
||||
self.assertEqual('refs/heads/stable', rrev)
|
||||
self.assertEqual('12345', lrev)
|
||||
|
||||
def test_verify_fails(self):
|
||||
"""Should fail when verification fails."""
|
||||
with mock.patch.object(self.wrapper, 'check_repo_verify', return_value=True):
|
||||
with mock.patch.object(self.wrapper, 'verify_rev', side_effect=Exception):
|
||||
with self.assertRaises(Exception):
|
||||
self.wrapper.check_repo_rev(self.GIT_DIR, 'stable')
|
||||
|
||||
def test_verify_ignore(self):
|
||||
"""Should pass when verification is disabled."""
|
||||
with mock.patch.object(self.wrapper, 'verify_rev', side_effect=Exception):
|
||||
rrev, lrev = self.wrapper.check_repo_rev(self.GIT_DIR, 'stable', repo_verify=False)
|
||||
self.assertEqual('refs/heads/stable', rrev)
|
||||
self.assertEqual(self.REV_LIST[1], lrev)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest.main()
|
||||
|
||||
8
tox.ini
8
tox.ini
@ -15,10 +15,11 @@
|
||||
# https://tox.readthedocs.io/
|
||||
|
||||
[tox]
|
||||
envlist = py36, py37, py38
|
||||
envlist = py27, py36, py37, py38
|
||||
|
||||
[gh-actions]
|
||||
python =
|
||||
2.7: py27
|
||||
3.6: py36
|
||||
3.7: py37
|
||||
3.8: py38
|
||||
@ -30,3 +31,8 @@ setenv =
|
||||
GIT_AUTHOR_NAME = Repo test author
|
||||
GIT_COMMITTER_NAME = Repo test committer
|
||||
EMAIL = repo@gerrit.nodomain
|
||||
|
||||
[testenv:py27]
|
||||
deps =
|
||||
mock
|
||||
pytest
|
||||
|
||||
Loading…
Reference in New Issue
Block a user