librsync  2.3.0
NEWS.md
1 # librsync NEWS
2 
3 ## librsync 2.3.0
4 
5 Released 2020-04-07
6 
7  * Bump minor version from 2.2.1 to 2.3.0 to reflect additional rs_sig_args()
8  and strong_len=-1 support.
9 
10  * Add public rs_sig_args() function for getting the recommend signature args
11  from the file size. Added support to rdiff for `--sum-size=-1` to indicate
12  "use minimum size safe against random block collisions". Added warning
13  output for sum-sizes that are too small to be safe. Fixed possible rdiff
14  bug affecting popt parsing on non-little-endian platforms. (dbaarda,
15  https://github.com/librsync/librsync/pull/109)
16 
17  * Fixed yet more compiler warnings for various platforms/compilers.
18  (Adsun701, texierp, https://github.com/librsync/librsync/pull/187,
19  https://github.com/librsync/librsync/pull/188)
20 
21  * Improved cmake popt handling to find popt dependencies using PkgConfig.
22  (ffontaine, https://github.com/librsync/librsync/pull/186)
23 
24  * Tidied internal code and improved tests for netint.[ch], tube.c, and
25  hashtable.h. (dbaarda, https://github.com/librsync/librsync/pull/183
26  https://github.com/librsync/librsync/pull/185).
27 
28  * Improved C99 compatibility. Add `-std=c99 -pedantic` to `CMAKE_C_FLAGS` for
29  gcc and clang. Fix all C99 warnings by making all code C99 compliant. Tidy
30  all CMake checks, #cmakedefines, and #includes. Fix 64bit support for
31  mdfour checksums (texierp, dbaarda,
32  https://github.com/librsync/librsync/pull/181,
33  https://github.com/librsync/librsync/pull/182)
34 
35  * Usage clarified in rdiff (1) man page. (AaronM04,
36  https://github.com/librsync/librsync/pull/180)
37 
38 ## librsync 2.2.1
39 
40 Released 2019-10-16
41 
42  * Fix #176 hangs calculating deltas for files larger than 4GB. (dbaarda,
43  https://github.com/librsync/librsync/pull/177)
44 
45 ## librsync 2.2.0
46 
47 Released 2019-10-12
48 
49  * Bump minor version from 2.1.0 to 2.2.0 to reflect additional RabinKarp
50  rollsum support.
51 
52  * Fix MSVC builds by adding missing LIBRSYNC_EXPORT to variables in
53  librsync.h, add -DLIBRSYNC_STATIC_DEFINE to the sumset_test target,
54  and correctly install .dll files in the bin directory.
55  (adsun701, https://github.com/librsync/librsync/pull/161)
56 
57  * Add RabinKarp rollsum support and make it the default. RabinKarp is a much
58  better rolling hash, which reduces the risk of hash collision corruption
59  and speeds up delta calculations. The rdiff cmd gets a new `-R
60  (rollsum|rabinkarp)` argument with the default being `rabinkarp`, Use `-R
61  rollsum` to generate backwards-compatible signatures. (dbaarda,
62  https://github.com/librsync/librsync/issues/3)
63 
64  * Use single-byte literal commands for small inserts in deltas. This makes
65  each small insert use 1 less byte in deltas. (dbaarda,
66  https://github.com/librsync/librsync/issues/120)
67 
68  * Fix multiple warnings (cross-)compiling for windows. (Adsun701,
69  https://github.com/librsync/librsync/pull/165,
70  https://github.com/librsync/librsync/pull/166)
71 
72  * Change rs_file_size() to report -1 instead of 0 for unknown file sizes (not
73  a regular file). (dbaarda https://github.com/librsync/librsync/pull/168)
74 
75  * Add cmake BUILD_SHARED_LIBS option for static library support.
76  BUILD_SHARED_LIBS defaults to ON, and can be set to OFF using `ccmake .` to
77  build librsync as a static library. (dbaarda
78  https://github.com/librsync/librsync/pull/169)
79 
80  * Fix compile errors and add .gitignore entries for MSVS 2019. Fixes
81  hashtable.h to be C99 compliant. (ardovm
82  https://github.com/librsync/librsync/pull/170)
83 
84 ## librsync 2.1.0
85 
86 Released 2019-08-19
87 
88  * Bump minor version from 2.0.3 to 2.1.0 to reflect additions to librsync.h.
89 
90  * Fix exporting of private symbols from librsync library. Add export of
91  useful large file functions `rs_file_open()`, `rs_file_close()`, and
92  `rs_file_size()` to librsync.h. Add export of `rs_signature_log_stats()` to
93  log signature hashtable hit/miss stats. Improve rdiff error output.
94  (dbaarda, https://github.com/librsync/librsync/issues/130)
95 
96  * Updated release process to include stable tarballs. (dbaarda,
97  https://github.com/librsync/librsync/issues/146)
98 
99  * Remove redundant and broken `--paranoia` argument from rdiff. (dbaarda,
100  https://github.com/librsync/librsync/issues/155)
101 
102  * Fix memory leak of `rs_signature_t->block_sigs` when freeing signatures.
103  (telles-simbiose, https://github.com/librsync/librsync/pull/147)
104 
105  * Document delta file format. (zmj,
106  https://github.com/librsync/librsync/issues/46)
107 
108  * Fix up doxygen comments. (dbaarda,
109  https://github.com/librsync/librsync/pull/151)
110 
111 ## librsync 2.0.2
112 
113 Released 2018-02-27
114 
115  * Improve CMake install paths configuration (wRAR,
116  https://github.com/librsync/librsync/pull/133) and platform support
117  checking when cross-compiling (fornwall,
118  https://github.com/librsync/librsync/pull/136).
119 
120  * Fix Unaligned memory access for rs_block_sig_init() (dbaarda,
121  https://github.com/librsync/librsync/issues/135).
122 
123  * Fix hashtable_test.c name collision for key_t in sys/types.h on some
124  platforms (dbaarda, https://github.com/librsync/librsync/issues/134)
125 
126  * Format code with consistent style, adding `make tidy` and `make
127  tidyc` targets for reformating code and comments. (dbaarda,
128  https://github.com/librsync/librsync/issues/125)
129 
130  * Removed perl as a build dependency. Note it is still required for some
131  tests. (dbaarda, https://github.com/librsync/librsync/issues/75)
132 
133  * Update RPM spec file for v2.0.2 and fix cmake man page install. (deajan,
134  https://github.com/librsync/librsync/issues/47)
135 
136 ## librsync 2.0.1
137 
138 Released 2017-10-17
139 
140  * Extensively reworked Doxygen documentation, now available at
141  http://librsync.sourcefrog.net/ (Martin Pool)
142 
143  * Removed some declarations from librsync.h that were unimplemented or no
144  longer ever useful: `rs_work_options`, `rs_accum_value`. Remove
145  declaration of unimplemented `rs_mdfour_file()`. (Martin Pool)
146 
147  * Remove shipped `snprintf` code: no longer acutally linked after changing to
148  CMake, and since it's part of C99 it should be widely available.
149  (Martin Pool)
150 
151  * Document that Ninja (http://ninja-build.org/) is supported under CMake.
152  It's a bit faster and nicer than Make. (Martin Pool)
153 
154  * `make check` (or `ninja check` etc) will now build and run the tests.
155  Previously due to a CMake limitation, `make test` would only run existing
156  tests and could fail if they weren't built.
157  (Martin Pool, https://github.com/librsync/librsync/issues/49)
158 
159  * Added cmake options to exclude rdiff target and compression from build.
160  See install documentation for details. Thanks to Michele Bertasi.
161 
162  * `popt` is only needed when `rdiff` is being built. (gulikoza)
163 
164  * Improved large file support for platforms using different variants
165  of `fseek` (`fseeko`, `fseeko64`, `_fseeki64`), `fstat` (`fstat64`,
166  `_fstati64`), and `fileno` (`_fileno`). (dbaarda, charlievieth,
167  gulikoza, marius-nicolae)
168 
169  * `rdiff -s` option now shows bytes read/written and speed. (gulikoza).
170  For delta operations it also shows hashtable match statistics. (dbaarda)
171 
172  * Running rdiff should not overwrite existing files (signatures, deltas and
173  new patched files) by default. If the destination file exists, rdiff will
174  now exit with an error. Add new option -f (--force) to overwrite existing
175  files. (gulikoza)
176 
177  * Improve signature memory allocation (doubling size instead of calling
178  realloc for every sig block) and added support for preallocation. See
179  streaming.md job->estimated_signature_count for usage when using the
180  library. `rdiff` uses this by default if possible. (gulikoza, dbaarda)
181 
182  * Significantly tidied signature handling code and testing, resulting in more
183  consistent error handling behaviour, and making it easier to plug in
184  alternative weak and strong sum implementations. Also fixed "slack delta"
185  support for delta calculation with no signature. (dbaarda)
186 
187  * `stdint.h` and `inttypes.h` from C99 is now required. Removed redundant
188  librsync-config.h header file. (dbaarda)
189 
190  * Lots of small fixes for windows platforms and building with MSVC.
191  (lasalvavida, mbrt, dbaarda)
192 
193  * New open addressing hashtable implementation that significantly speeds up
194  delta operations, particularly for large files. Also fixed degenerate
195  behaviour with large number of duplicate blocks like runs of zeros
196  in sparse files. (dbaarda)
197 
198  * Optional support with cmake option for using libb2 blake2 implementation.
199  Also updated included reference blake2 implementation with bug fixes
200  (dbaarda).
201 
202  * Improved default values for input and output buffer sizes. The defaults are
203  now --input-size=0 and --output-size=0, which will choose recommended
204  default sizes based on the --block-size and the operation being performed.
205  (dbaarda)
206 
207  * Fixed hanging for truncated input files. It will now correctly report an
208  error indicating an unexpected EOF was encountered. (dbaarda,
209  https://github.com/librsync/librsync/issues/32)
210 
211  * Fixed #13 so that faster slack delta's are used for signatures of
212  empty files. (dbaarda,
213  https://github.com/librsync/librsync/issues/13)
214 
215  * Fixed #33 so rs_job_iter() doesn't need calling twice with eof=1.
216  Also tidied and optimized it a bit. (dbaarda,
217  https://github.com/librsync/librsync/issues/33)
218 
219  * Fixed #55 remove excessive rs_fatal() calls, replacing checks for
220  programming errors with assert statements. Now rs_fatal() will only
221  be called for rare unrecoverable fatal errors like malloc failures or
222  impossibly large inputs. (dbaarda,
223  https://github.com/librsync/librsync/issues/55)
224 
225 ## librsync 2.0.0
226 
227 Released 2015-11-29
228 
229 Note: despite the major version bump, this release has few changes and should
230 be binary and API compatible with the previous version.
231 
232  * Bump librsync version number to 2.0, to match the library
233  soname/dylib version.
234  (Martin Pool, https://github.com/librsync/librsync/issues/48)
235 
236 ## librsync 1.0.1 (2015-11-21)
237 
238  * Better performance on large files. (VictorDenisov)
239 
240  * Add comment on usage of rs_build_hash_table(), and assert correct use.
241  Callers must call rs_build_hash_table() after loading the signature,
242  and before calling rs_delta_begin().
243  Thanks to Paul Harris <paulharris@computer.org>
244 
245  * Switch from autoconf to CMake.
246 
247  Thanks to Adam Schubert.
248 
249 ## librsync 1.0.0 (2015-01-23)
250 
251  * SECURITY: CVE-2014-8242: librsync previously used a truncated MD4
252  "strong" check sum to match blocks. However, MD4 is not cryptographically
253  strong. It's possible that an attacker who can control the contents of one
254  part of a file could use it to control other regions of the file, if it's
255  transferred using librsync/rdiff. For example this might occur in a
256  database, mailbox, or VM image containing some attacker-controlled data.
257 
258  To mitigate this issue, signatures will by default be computed with a
259  256-bit BLAKE2 hash. Old versions of librsync will complain about a
260  bad magic number when given these signature files.
261 
262  Backward compatibility can be obtained using the new
263  `rdiff sig --hash=md4`
264  option or through specifying the "signature magic" in the API, but
265  this should not be used when either the old or new file contain
266  untrusted data.
267 
268  Deltas generated from those signatures will also use BLAKE2 during
269  generation, but produce output that can be read by old versions.
270 
271  See https://github.com/librsync/librsync/issues/5
272 
273  Thanks to Michael Samuel <miknet.net> for reporting this and offering an
274  initial patch.
275 
276  * Various build fixes, thanks Timothy Gu.
277 
278  * Improved rdiff man page from Debian.
279 
280  * Improved librsync.spec file for building RPMs.
281 
282  * Fixed bug #1110812 'internal error: job made no progress'; on large
283  files.
284 
285  * Moved hosting to https://github.com/librsync/librsync/
286 
287  * Travis-CI.org integration test at https://travis-ci.org/librsync/librsync/
288 
289  * You can set `$LIBTOOLIZE` before running `autogen.sh`, for example on
290  OS X Homebrew where it is called `glibtoolize`.
291 
292 ## 0.9.7 (released 2004-10-10)
293 
294  * Yet more large file support fixes.
295 
296  * `extern "C"` guards in librsync.h to let it be used from C++.
297 
298  * Removed Debian files from dist tarball.
299 
300  * Changed rdiff to an installed program on "make install".
301 
302  * Refactored delta calculation code to be cleaner and faster.
303 
304  * \#879763: Fixed mdfour to work on little-endian machines which don't
305  like unaligned word access. This should make librsync work on
306  pa-risc, and it makes it slightly faster on ia64.
307 
308  * \#1022764: Fix corrupted encoding of some COPY commands in large
309  files.
310 
311  * \#1024881: Print long integers directly, rather than via casts to
312  double.
313 
314  * Fix printf formats for size_t: both the format and the argument
315  should be cast to long.
316 
317 ## 0.9.6
318 
319  * Large file support fixes.
320 
321  * [v]snprintf or _[v]snprintf autoconf replacement function fix.
322 
323  * Changed installed include file from rsync.h to librsync.h.
324 
325  * Migration to sourceforge for hosting.
326 
327  * Rollsum bugfix that produces much smaller deltas.
328 
329  * Memory leaks bugfix patches.
330 
331  * mdfour bigendian and >512M bugfix, plus optimisations patch.
332 
333  * autoconf/automake updates and cleanups for autoconf 2.53.
334 
335  * Windows compilation patch, heavily modified.
336 
337  * MacOSX compilation patch, modified to autoconf vararg macro fix.
338 
339  * Debian package build scripts patch.
340 
341 ## 0.9.5
342 
343  * Bugfix patch from Shirish Hemant Phatak
344 
345 ## 0.9.4: (library 1.1.0)
346 
347  * Fixes for rsync.h from Thorsten Schuett <thorsten.schuett@zib.de>
348 
349  * RLL encoding fix from Shirish Hemant Phatak <shirish@nustorage.com>
350 
351  * RPM spec file by Peter J. Braam <braam@clusterfs.com>
352 
353  * No (intentional) changes to binary API.
354 
355 ## 0.9.3
356 
357  * Big speed improvements in MD4 routines and generation of weak
358  checksums.
359 
360  * Patch to build on FreeBSD by Jos Backus <josb@cncdsl.com>
361 
362  * Suggestions to build on Solaris 2.6 from Alberto Accomazzi
363  <aaccomazzi@cfa.harvard.edu>
364 
365  * Add rs_job_drive, a generic mechanism for turning the library into
366  blocking mode. rs_whole_run now builds on top of this. The
367  filebuf interface has changed a little to accomodate it.
368 
369  * Generating and loading signatures now generates statistics.
370 
371  * More test cases.
372 
373  * I suspect there may be a bug in rolling checksums, but it probably
374  only causes inefficiency and not corruption.
375 
376  * Portability fixes for alphaev67-dec-osf5.1; at the moment builds
377  but does not work because librsync tries to do unaligned accesses.
378 
379  * Works on sparc64-unknown-linux-gnu (Debian/2.2)
380 
381 ## 0.9.2
382 
383  * Improve delta algorithm so that deltas are actually
384  delta-compressed, rather than faked.
385 
386 ## 0.9.1
387 
388  * Rename the library to `librsync'.
389 
390  * Portability fixes.
391 
392  * Include the popt library, and use it to build rdiff if the library
393  is not present on the host.
394 
395  * Add file(1) magic for rdiff.
396 
397  * Add more to the manual pages.
398 
399  * It's no longer necessary to call rs_buffers_init on a stream before
400  starting to use it: all the internal data is kept in the job, not
401  in the stream.
402 
403  * Rename rs_stream_t to rs_buffers_t, a more obvious name. Pass the
404  buffers to every rs_job_iter() call, rather than setting it at
405  startup. Similarly for all the _begin() functions.
406 
407  * rs_job_new also takes the initial state function.
408 
409  * Return RS_PARAM_ERROR when library is misused.
410 
411 ## 0.9.0
412 
413  * Redesign API to be more like zlib/bzlib.
414 
415  * Put all command-line functions into a single rdiff(1) program.
416 
417  * New magic number `rs6'
418 
419  * Change to using popt for command line parsing.
420 
421  * Use Doxygen for API documentation.
422 
423 ## 0.5.7
424 
425  * Changes stats string format.
426 
427  * Slightly improved test cases
428 
429 ## 0.5.6
430 
431  * Don't install debugging tools into /usr/local/bin; leave them in
432  the source directory.
433 
434  * Fix libhsync to build on (sgi-mips, IRIX64, gcc, GNU Make)
435 
436  * Include README.CVS in tarball
437 
438  * Back out of using libtool and shared libraries, as it is
439  unnecessary at this stage, complicates installation and slows down
440  compilation.
441 
442  * Use mapptr when reading data to decode, so that decoding should
443  have less latency and be more reliable.
444 
445  * Cope better on systems that are missing functions like snprintf.
446 
447 ## 0.5.5
448 
449  * Put genuine search encoding back into the nad algorithm, and
450  further clean up the nad code. Literals are now sent out using a
451  literal buffer integrated with the input mapptr so that data is not
452  copied. Checksums are still calculated from scratch each time
453  rather than by rolling -- this is very slow but simple.
454 
455  * Reshuffle test cases so that they use files generated by hsmapread,
456  rather than the source directory. This makes the tests quicker and
457  more reproducible, hopefully without losing coverage. Further
458  develop the test driver framework.
459 
460  * Add hsdumpsums debugging tool.
461 
462  * Hex strings (eg strong checksums) are broken up by underscores for
463  readability.
464 
465  * Stats now go to the log rather than stdout.
466 
467  * mapptr acts properly when we're skipping/rewinding to data already
468  present in the buffer -- it does a copy if required, but not
469  necessarily real IO.
470 
471 ## 0.5.4
472 
473  * Improved mapptr input code
474 
475  * Turn on more warnings if using gcc
476 
477  * More test cases
478 
479 ## 0.5.3
480 
481  * Improvements to mapptr to make it work better for network IO.
482 
483  * Debug trace code is compiled in unless turned off in ./configure
484  (although most programs will not write it out unless asked.)
485 
486  * Add libhsyncinfo program to show compiled-in settings and version.
487 
488  * Add test cases that run across localhost TCP sockets.
489 
490  * Improved build code; should now build easily from CVS through
491  autogen.sh.
492 
493  * Improved trace code.
494 
495  * Clean up to build on sparc-sun-solaris2.8, and in the process clean
496  up the handling of bytes vs chars, and of building without gcc
497 
498  * Reverse build scripts so that driver.sh calls the particular
499  script.
500 
501 ## 0.5.2
502 
503  * Use mapptr for input.
504 
505  * Implement a new structure for encoding in nad.c. It doesn't
506  encode at the moment, but it's much more maintainable.
507 
508  * More regression cases.
509 
510  * Clean up build process.
511 
512 ## 0.5.0
513 
514  * Rewrite hs_inbuf and hs_encode to make them simpler and more
515  reliable.
516 
517  * Test cases for input handling.
518 
519  * Use the map_ptr idea for input from both streams and files.
520 
521 ## 0.4.1
522 
523  * automake/autoconf now works cleanly when the build directory is
524  different to the source directory.
525 
526  * --enable-ccmalloc works again.
527 
528 ## 0.4.0
529 
530 * A much better regression suite.
531 
532 * CHECKSUM token includes the file's checksum up to the current
533  location, to aid in self-testing.
534 
535 * Various bug fixes, particularly to do with short IO returns.
Description of input and output buffers.
Definition: librsync.h:322
rs_result rs_job_iter(rs_job_t *job, rs_buffers_t *buffers)
Run a rs_job state machine until it blocks (RS_BLOCKED), returns an error, or completes (RS_DONE)...
Definition: job.c:93
Bad value passed in to library, probably an application bug.
Definition: librsync.h:200
void * block_sigs
The packed block_sigs for all blocks.
Definition: sumset.h:43
LIBRSYNC_EXPORT int rs_file_close(FILE *file)
Close a file with special handling for stdin or stdout.
Definition: fileutil.c:115
Signature of a whole file.
Definition: sumset.h:37
LIBRSYNC_EXPORT FILE * rs_file_open(char const *filename, char const *mode, int force)
Open a file with special handling for stdin or stdout.
Definition: fileutil.c:76
LIBRSYNC_EXPORT void rs_signature_log_stats(rs_signature_t const *sig)
Log the rs_signature_delta match stats.
Definition: sumset.c:259
LIBRSYNC_EXPORT rs_result rs_sig_args(rs_long_t old_fsize, rs_magic_number *magic, size_t *block_len, size_t *strong_len)
Get or check signature arguments for a given file size.
Definition: sumset.c:117
The hashtable type.
Definition: hashtable.h:128
rs_result rs_job_drive(rs_job_t *job, rs_buffers_t *buf, rs_driven_cb in_cb, void *in_opaque, rs_driven_cb out_cb, void *out_opaque)
Actively process a job, by making callbacks to fill and empty the buffers until the job is done...
Definition: job.c:152
LIBRSYNC_EXPORT rs_long_t rs_file_size(FILE *file)
Get the size of a file.
Definition: fileutil.c:122