Similar to the `git_commit_create_from_...` APIs, a simple amend
function that uses smart defaults and amends HEAD from the staged
changes or a given tree.
Similar to the `git_commit_create_from_stage` function, a batteries-
included commit creation function that uses smart defaults and commits a
given tree.
Introduce `git_commit_create_ext` and an options structure for
extensibility. Non-default arguments (message encoding, reference
updates) are in the options structure.
This simplifies the complex self-service commit creation APIs and allows
for future sustainable improvements.
Ye olde PCRE (8.45) was end-of-lifed several years ago. For our bundled
regular expression implementation, we want to include the new,
still-maintained PCRE2 framework. Include PCRE2 v10.47.
This was caused by the tests being compiled with -D_FILE_OFFSET_BITS=64
which causes incompatibilities if libgit2 itself is not.
As this has been resolved the environment variable is no longer
necessary and can be removed.
On 32bit systems the git_fs_path_lstat from libgit2 by default uses 32bit stat
structs while the tests are being compiled with struct stat64 via
_FILE_OFFSET_BITS=64.
This discrepancy causes the "flaky" stat failures in tests.
The solution is to use the same _FILE_OFFSET_BITS as the library by
setting _FILE_OFFSET_BITS globally
Co-authored-by: Edward Thomson <ethomson@edwardthomson.com>
Make a distinction between generated headers and "translated" headers.
This is important to support build-time dependencies when headers are
updated.
Generated headers are those which contain build-time feature
specifications, like `git2_features.h` that are internal to the build
and `experimental.h` that contain API information.
Translated headers are the headers that are in `include/git2`, but may
be translated to have a unique prefix like `incklude/git2-experimental`.
This distinction is important so that the CMakeFiles.txt depend on the
in-tree include files (`src/include`) and the generated header files
_but not_ the translated header files. Otherwise there are two `pack.h`
and it's unclear whether the in-tree build is targeting the one in
`src/include` or the one in the build tree.
Without this, updating an in-tree header file like `pack.h` would not
cause a rebuild of its dependencies.
The implementation here seems to be sort of a copy
from the reference impl in RFC 6234 [2].
When multiple threads hash concurrently,
they race on this shared static variable.
It then corrupts the length-overflow detection,
and produces incorrect SHA-256 digests.
Here we replace it with a `static` function with a local variable.
The bug only affects the `GIT_SHA256_BUILTIN` backend.
The SHA-1 code path uses `sha1dc` which does not have this issue.
Reproducer:
```c
#include <stddef.h>
#include <stdio.h>
#include <string.h>
#include <pthread.h>
#include <git2.h>
#define NUM_THREADS 8
#define ITERATIONS 100000
static volatile int found_bug = 0;
void *hash_thread(void *arg) {
int id = *(int *)arg;
const char *data = "hello world\n";
size_t len = strlen(data);
git_object_id_options opts = GIT_OBJECT_ID_OPTIONS_INIT;
opts.object_type = GIT_OBJECT_BLOB;
opts.oid_type = GIT_OID_SHA256;
git_oid reference, result;
git_object_id_from_buffer(&reference, data, len, &opts);
for (int i = 0; i < ITERATIONS && !found_bug; i++) {
git_object_id_from_buffer(&result, data, len, &opts);
if (!git_oid_equal(&reference, &result)) {
found_bug = 1;
printf("BUG: thread %d, iteration %d\n", id, i);
break;
}
}
return NULL;
}
int main(void) {
git_libgit2_init();
pthread_t threads[NUM_THREADS];
int ids[NUM_THREADS];
for (int i = 0; i < NUM_THREADS; i++) {
ids[i] = i;
pthread_create(&threads[i], NULL, hash_thread, &ids[i]);
}
for (int i = 0; i < NUM_THREADS; i++)
pthread_join(threads[i], NULL);
if (!found_bug)
printf("No bug triggered\n");
git_libgit2_shutdown();
return found_bug ? 1 : 0;
}
```
Build and run (from libgit2 repo root):
```sh
mkdir build && cd build
cmake .. -DEXPERIMENTAL_SHA256=ON -DUSE_SHA256=Builtin \
-DUSE_HTTPS=OFF -DUSE_SSH=OFF -DUSE_NTLMCLIENT=OFF \
-DBUILD_SHARED_LIBS=OFF -DCMAKE_BUILD_TYPE=Debug
make libgit2package
cd ..
cc -O0 -pthread -DGIT_EXPERIMENTAL_SHA256=1 \
-I include -o repro repro.c \
build/libgit2-experimental.a -lz -lpcre2-8
./repro
```
See <https://github.com/rust-lang/git2-rs/issues/1255> for more.
[1]: https://github.com/libgit2/libgit2/blob/1affb8b19/src/util/hash/rfc6234/sha224-256.c#L86-L91
[2]: https://www.rfc-editor.org/rfc/rfc6234#section-8.2.2
