Our file utils functions all have a "futils" prefix, e.g.
`git_futils_touch`. One would thus naturally guess that their
definitions and implementation would live in files "futils.h" and
"futils.c", respectively, but in fact they live in "fileops.h".
Rename the files to match expectations.
With commit dedf70ad2 (patch_parse: do not depend on parsed buffer's
lifetime, 2019-07-05), all lines of the patch are allocated with
`strdup` to make lifetime of the parsed patch independent of the buffer
that is currently being parsed. In patch b08932824 (patch_parse: ensure
valid patch output with EOFNL, 2019-07-11), we introduced another
code location where we add lines to the parsed patch. But as that one
was implemented via a separate pull request, it wasn't converted to use
`strdup`, as well. As a consequence, we generate a segfault when trying
to deallocate the potentially static buffer that's now in some of the
lines.
Use `git__strdup` to fix the issue.
When using templates to initialize a git repository, then git-init(1)
will copy over all contents of the template directory. These will be
preferred over the default ones created by git-init(1). While we mostly
do the same, there is the exception of "HEAD". While we do copy over the
template's HEAD file, afterwards we'll immediately re-initialize its
contents with either the default "ref: refs/origin/master" or the init
option's `initial_head` field.
Let's fix the inconsistency with upstream git-init(1) by not overwriting
the template HEAD, but only if the user hasn't set `opts.initial_head`.
If the `initial_head` field has been supplied, we should use that
indifferent from whether the template contained a HEAD file or not. Add
tests to verify we correctly use the template directory's HEAD file and
that `initial_head` overrides the template.
Update `git_repository_init_ext` to use our typical style of error
handling. The function had multiple statements which didn't `goto out`
immediately but instead deferred it to later calls combined with `if`
statements.
The error handling in `git_repository_create_head` completely swallows
all error codes. While probably not too much of a problem, this also
violates our usual coding style.
Refactor the code to use a local `error` variable with the typical `goto
out` statements.
All tests in repo::template have a common pattern of first setting up
templates, then settung up the repository that makes use of those
templates via several init options. Refactor this pattern into two
functions `setup_templates` and `setup_repo` that handle most of that
logic to make it easier to spot what a test actually wants to check.
Furthermore, this also refactors how we clean up after the tests.
Previously, it was a combination of manually calling
`cl_fixture_cleanup` and `cl_set_cleanup`, which really is kind of hard
to read. This commit refactors this to instead provide the cleaning
parameters in the setup functions. All cleanups are then performed in
the suite's cleanup function.
There's quite a lot of supporting code for our templates and they are an
obvious standalone feature. Thus, let's extract those tests into their
own suite to also make refactoring of them easier.
With the introduction of data variants for suites, we started
undercounting the number of suites as we didn't account for those that
were executed twice. This was then adjusted to count the number of
initializers instead, but this fails to account for suites without any
initializers at all.
Fix the suite count by counting either the number of initializers or, if
there is no initializer, count it as a single suite, only.
We currently have no job that compiles libgit2 with the WinHTTP backend
for SHA1. Due to this, a compile error has been introduced and not
noticed for several months. Change the x86 MSVC job to use the HTTPS
backend for SHA1. The x86 job was chosen with no particular reason.
In commit bbf034ab9 (hash: move `git_hash_prov` into Win32 backend,
2019-02-22), the `git_hash_prov`'s structure name has been removed in
favour of its typedef'ed name. But as we have no CI that compiles with
the WinHTTPS hashing backend right now, it wasn't noticed that the
implementation that uses this struct wasn't changed correctly.
Fix the struct type to make it compile again.
When selecting the SHA1 backend, we only include the respective C
implementation of the selected backend. But since commit bd48bf3fb
(hash: introduce source files to break include circles, 2019-06-14), we
have introduced separate headers and compilation units for all hashes.
So by not including the headers, we may not honor them to compute
whether a file needs to be recompiled and they also will not be
displayed in IDEs.
Add the header files to fix this problem.
When computing whether we need to store a negative pattern, we iterate
through all previously known patterns and check whether the negative
pattern undoes any of the previous ones. In doing so we call `wildmatch`
and check it's return for any negative error values. If there was a
negative return, we will abort and bubble up that error to the caller.
In fact, this check for negative values stems from the time where we
still used `fnmatch` instead of `wildmatch`. For `fnmatch`, negative
values indicate a "real" error, while for `wildmatch` a negative value
may be returned if the matching was prematurely aborted. A premature
abort may for example also happen if the pattern matches a prefix of the
haystack if the pattern is shorter. Returning an error in that case is
the wrong thing to do.
Fix the code to compare for equality with `WM_MATCH`, only. Negative
values returned by `wildmatch` are perfectly fine and thus should be
ignored. Add a test that verifies we do not see the error.
When our object cache is full, we pick eight items (or the whole cache,
if there are fewer) and evict them. For small cache sizes, this is fine,
but when we're dealing with a large number of objects, we can repeatedly
exhaust the cache and spend a large amount of time in git_oidmap_iterate
trying to find items to evict.
Instead, let's assume that if the cache gets full, we have a large
number of objects that we're handling, and be more aggressive about
evicting items. Let's remove one item for every 2048 items, but not less
than 8. This causes us to scale our evictions in proportion to the size
of the cache and significantly reduces the time we spend in
git_oidmap_iterate.
Before this change, a full pack of all the non-blob objects in the Linux
repository took in excess of 30 minutes and spent 62.3% of total runtime
in odb_read_1 and its children, and 44.3% of the time in
git_oidmap_iterate. With this change, the same operation now takes 14
minutes and 44 seconds, and odb_read_1 accounts for only 35.9% of total
time, whereas git_oidmap_iterate consists of 6.2%.
Note that we do spend a little more time inflating objects and a decent
amount more time in memcmp. However, overall, the time taken is
significantly improved, and time in pack building is now dominated by
git_delta_create_from_index (33.7%), which is what we would expect.
The packbuilder code allocates memory in chunks. When it needs to
allocate, it tries to add 1024 to the number of objects and multiply by
3/2. However, it actually multiplies by 1 instead, since it performs an
integral division in the expression "3 / 2" and only then multiplies by
the increased number of objects.
The current behavior causes the code to waste massive amounts of time
copying memory when it reallocates, causing inserting all non-blob
objects in the Linux repository into a new pack to take some
indeterminate time greater than 5 minutes instead of 52 seconds.
Correct this error by first dividing by two, and only then multiplying
by 3. We still check for overflow for the multiplication, which is the
only part that can overflow. This appears to be the only place in the
code base which has this problem.
Right now, we are unconditionally applying all macros found in a
gitatttributes file. But quoting gitattributes(5):
Custom macro attributes can be defined only in top-level
gitattributes files ($GIT_DIR/info/attributes, the .gitattributes
file at the top level of the working tree, or the global or
system-wide gitattributes files), not in .gitattributes files in
working tree subdirectories. The built-in macro attribute "binary"
is equivalent to:
So gitattribute files in subdirectories of the working tree may
explicitly _not_ contain macro definitions, but we do not currently
enforce this limitation.
This patch introduces a new parameter to the gitattributes parser that
tells whether macros are allowed in the current file or not. If set to
`false`, we will still parse macros, but silently ignore them instead of
adding them to the list of defined macros. Update all callers to
correctly determine whether the to-be-parsed file may contain macros or
not. Most importantly, when walking up the directory hierarchy, we will
only set it to `true` once it reaches the root directory of the repo
itself.
Add a test that verifies that we are indeed not applying macros from
subdirectories. Previous to these changes, the test would've failed.
The gitattributes code is one of our oldest and most-untouched codebases
in libgit2, and as such its code style doesn't quite match our current
best practices. Refactor the function `git_attr_file__parse_buffer` to
better match them.
The gitattributes code is one of our oldest and most-untouched codebases
in libgit2, and as such its code style doesn't quite match our current
best practices. Refactor the function `git_attr_file__lookup_standalone`
to better match them.
A macro without any assignments is considered an invalid macro by the
attributes cache and is thus not getting added to the macro map at all.
But as `git_attr_cache__insert_macro` returns success with neither
free'ing nor adopting the macro into its map, this will cause a memory
leak.
Fix this by freeing the macro in the function if it's not going to be
added. This is perfectly fine to do, as callers assume that the
attrcache will have the macro adopted on success anyway.
The function `git_attr_cache__insert_macro` is responsible for adopting
macros in the per-repo macro cache. When adding a macro that replaces an
already existing macro (e.g. because of re-parsing gitattributes files),
then we do not free the previous macro and thus cause a memory leak.
Fix this leak by first checking if the cache already has a macro defined
with the same name. If so, free it before replacing the cache entry with
the new instance.
As macros are a specific functionality in the gitattributes code, it
makes sense to extract them into their own test suite, too. This makes
finding macro-related tests easier.
In commit 45f24e787 (git_repository_init: stop traversing at
windows root, 2019-04-12), we have fixed `git_futils_mkdir` to
correctly handle the case where we create a directory in
Windows-style filesystem roots like "C:\repo".
The problem here is an off-by-one: previously, to that commit,
we've been checking wether the parent directory's length is equal
to the root directory's length incremented by one. When we call
the function with "/example", then the parent directory's length
("/") is 1, but the root directory offset is 0 as the path is
directly rooted without a drive prefix. This resulted in `1 == 0 +
1`, which was true. With the change, we've stopped incrementing
the root directory length, and thus now compare `1 <= 0`, which
is false.
The previous way of doing it was kind of finicky any non-obvious,
which is also why the error was introduced. So instead of just
re-adding the increment, let's explicitly add a condition that
aborts finding the parent if the current parent path is "/".
Making this change causes Azure Pipelines to fail the testcase
repo::init::nonexistent_paths on Unix-based systems. This is because we
have just fixed creating directories in the filesystem root, which
previously didn't work. As Docker-based tests are running as root user,
we are thus able to create the non-existing path and will now succeed to
create the repository that was expected to actually fail.
Let's split this up into three different tests:
- A test to verify that we do not create repos in a non-existing parent
directoy if the flag `GIT_REPOSITORY_INIT_MKPATH` is not set.
- A test to verify that we fail if the root directory does not exist. As
there is a common root directory on Unix-based systems that always
exist, we can only test for this on Windows-based systems.
- A test to verify that we fail if trying to create a repository in an
unwriteable parent directory. We can only test this if not running
tests as root user, as CAP_DAC_OVERRIDE will cause us to ignore
permissions when creating files.
When either the old or new file contents have no newline at the end of
the file, then git-diff(1) will print out a "\ No newline at end of
file" indicator. While we do correctly handle this in the case where the
new file has this indcator, we fail to parse patches where the old file
is missing a newline at EOF.
Fix this bug by handling and missing newline indicators in the old file.
Add tests to verify that we can parse such files.
The patch ID is supposed to be mostly context-insignificant and
thus only includes added or deleted lines. As such, we shouldn't honor
end-of-file-without-newline markers in diffs.
Ignore such lines to fix how we compute the patch ID for such diffs.
Right now, all configuration file backends are expected to
directly mess with the configuration parser's internals in order
to set it up. Let's avoid doing that by implementing both a
`git_config_parser_init` and `git_config_parser_dispose` function
to clearly define the interface between configuration backends
and the parser.
Ideally, we would make the `git_config_parser` structure
definition private to its implementation. But as that would
require an additional memory allocation that was not required
before we just live with it being visible to others.
