Joris van Rantwijk
a6fe2a199c
Add some automated tests
2022-07-03 15:59:00 +02:00
Joris van Rantwijk
35ae98cc65
Use fallocate() instead of posix_fallocate()
...
On systems that do not support fallocate() (NFS, BeeGFS),
posix_fallocate() emulates the allocation by writing zeros
which is extremely inefficient.
In these cases we would rather skip-preallocation and
simply use ftruncate() to set the file size without allocating.
2022-07-03 10:40:32 +02:00
Joris van Rantwijk
5c24788b09
On invalid parameters, print short help message
2022-07-03 09:29:27 +02:00
Joris van Rantwijk
2e8dae2245
Use nullptr instead of NULL
2022-07-03 01:21:40 +02:00
Joris van Rantwijk
5b2a06aabe
Implement background I/O for merge passes
2022-07-03 00:44:58 +02:00
Joris van Rantwijk
4f6b76e785
Implement background I/O for the sorting pass
2022-07-01 23:47:39 +02:00
Joris van Rantwijk
57c20dbca3
Add command-line option for I/O thread
...
Actual background I/O not yet implemented.
2022-07-01 20:46:46 +02:00
Joris van Rantwijk
eb3b06eede
recgen: Use salt when generating duplicate records
2022-06-27 20:53:04 +02:00
Joris van Rantwijk
c47ff98883
Use parallel quicksort for multi-pass sorts
2022-06-27 20:53:00 +02:00
Joris van Rantwijk
8114068ca7
recgen: Remove debug output
2022-06-26 20:55:34 +02:00
Joris van Rantwijk
2977f50539
Use multi-threaded quicksort for single-pass sorts
2022-06-26 20:51:35 +02:00
Joris van Rantwijk
ae33feaca4
recgen: Improve accuracy of duplicate fraction
2022-06-26 13:15:26 +02:00
Joris van Rantwijk
b091d8b1eb
Avoid unnecessary I/O during merging
2022-06-26 10:46:36 +02:00
Joris van Rantwijk
24290acc9c
Fix README
2022-06-25 17:20:43 +02:00
Joris van Rantwijk
0696549e5e
Add README
2022-06-25 17:09:37 +02:00
Joris van Rantwijk
d40174327e
Add comments from reference impl of xoroshiro128+
2022-06-25 15:15:57 +02:00
Joris van Rantwijk
97bc26ce7f
Move source files to subdirectory
2022-06-25 13:57:12 +02:00
Joris van Rantwijk
67be4cbf7b
Use 'M' or 'G' suffix for --memory option
2022-06-25 13:33:41 +02:00
Joris van Rantwijk
771394aa9d
Avoid O_TMPFILE, instead just unlink() explicitly
...
O_TMPFILE is not universally supported on all filesystems.
2022-06-25 13:10:06 +02:00
Joris van Rantwijk
99458a6b28
recgen: Add explicit random seed option
2022-06-25 11:58:28 +02:00
Joris van Rantwijk
ac37e10e70
Fix bug in handling of unbalanced merge tree
2022-06-25 09:01:22 +02:00
Joris van Rantwijk
f490e46b0f
Add gitignore
2022-06-24 16:52:56 +02:00
Joris van Rantwijk
c1d088a3c7
Fix bugs in multi-pass sorting
2022-06-24 16:31:09 +02:00
Joris van Rantwijk
c9ea115e68
Add tool to generate random records
2022-06-24 16:30:06 +02:00
Joris van Rantwijk
9b303bb2f3
Fix comments
2022-06-24 15:14:51 +02:00
Joris van Rantwijk
af0d74dc21
Implement duplicate filtering - not yet tested
2022-06-22 22:45:09 +02:00
Joris van Rantwijk
1da4983c7d
Implement merge pass - not yet tested
2022-06-22 22:17:14 +02:00
Joris van Rantwijk
5b5c090bc5
Current snapshot, work-in-progress
2022-06-22 22:14:47 +02:00
Joris van Rantwijk