1
0
Fork 0
Commit Graph

28 Commits

Author SHA1 Message Date
Joris van Rantwijk a6fe2a199c Add some automated tests 2022-07-03 15:59:00 +02:00
Joris van Rantwijk 35ae98cc65 Use fallocate() instead of posix_fallocate()
On systems that do not support fallocate() (NFS, BeeGFS),
posix_fallocate() emulates the allocation by writing zeros
which is extremely inefficient.

In these cases we would rather skip-preallocation and
simply use ftruncate() to set the file size without allocating.
2022-07-03 10:40:32 +02:00
Joris van Rantwijk 5c24788b09 On invalid parameters, print short help message 2022-07-03 09:29:27 +02:00
Joris van Rantwijk 2e8dae2245 Use nullptr instead of NULL 2022-07-03 01:21:40 +02:00
Joris van Rantwijk 5b2a06aabe Implement background I/O for merge passes 2022-07-03 00:44:58 +02:00
Joris van Rantwijk 4f6b76e785 Implement background I/O for the sorting pass 2022-07-01 23:47:39 +02:00
Joris van Rantwijk 57c20dbca3 Add command-line option for I/O thread
Actual background I/O not yet implemented.
2022-07-01 20:46:46 +02:00
Joris van Rantwijk eb3b06eede recgen: Use salt when generating duplicate records 2022-06-27 20:53:04 +02:00
Joris van Rantwijk c47ff98883 Use parallel quicksort for multi-pass sorts 2022-06-27 20:53:00 +02:00
Joris van Rantwijk 8114068ca7 recgen: Remove debug output 2022-06-26 20:55:34 +02:00
Joris van Rantwijk 2977f50539 Use multi-threaded quicksort for single-pass sorts 2022-06-26 20:51:35 +02:00
Joris van Rantwijk ae33feaca4 recgen: Improve accuracy of duplicate fraction 2022-06-26 13:15:26 +02:00
Joris van Rantwijk b091d8b1eb Avoid unnecessary I/O during merging 2022-06-26 10:46:36 +02:00
Joris van Rantwijk 24290acc9c Fix README 2022-06-25 17:20:43 +02:00
Joris van Rantwijk 0696549e5e Add README 2022-06-25 17:09:37 +02:00
Joris van Rantwijk d40174327e Add comments from reference impl of xoroshiro128+ 2022-06-25 15:15:57 +02:00
Joris van Rantwijk 97bc26ce7f Move source files to subdirectory 2022-06-25 13:57:12 +02:00
Joris van Rantwijk 67be4cbf7b Use 'M' or 'G' suffix for --memory option 2022-06-25 13:33:41 +02:00
Joris van Rantwijk 771394aa9d Avoid O_TMPFILE, instead just unlink() explicitly
O_TMPFILE is not universally supported on all filesystems.
2022-06-25 13:10:06 +02:00
Joris van Rantwijk 99458a6b28 recgen: Add explicit random seed option 2022-06-25 11:58:28 +02:00
Joris van Rantwijk ac37e10e70 Fix bug in handling of unbalanced merge tree 2022-06-25 09:01:22 +02:00
Joris van Rantwijk f490e46b0f Add gitignore 2022-06-24 16:52:56 +02:00
Joris van Rantwijk c1d088a3c7 Fix bugs in multi-pass sorting 2022-06-24 16:31:09 +02:00
Joris van Rantwijk c9ea115e68 Add tool to generate random records 2022-06-24 16:30:06 +02:00
Joris van Rantwijk 9b303bb2f3 Fix comments 2022-06-24 15:14:51 +02:00
Joris van Rantwijk af0d74dc21 Implement duplicate filtering - not yet tested 2022-06-22 22:45:09 +02:00
Joris van Rantwijk 1da4983c7d Implement merge pass - not yet tested 2022-06-22 22:17:14 +02:00
Joris van Rantwijk 5b5c090bc5 Current snapshot, work-in-progress 2022-06-22 22:14:47 +02:00