1Using talloc in Samba4 2---------------------- 3 4Andrew Tridgell 5September 2004 6 7The most current version of this document is available at 8 http://samba.org/ftp/unpacked/samba4/source/lib/talloc/talloc_guide.txt 9 10If you are used to talloc from Samba3 then please read this carefully, 11as talloc has changed a lot. 12 13The new talloc is a hierarchical, reference counted memory pool system 14with destructors. Quite a mounthful really, but not too bad once you 15get used to it. 16 17Perhaps the biggest change from Samba3 is that there is no distinction 18between a "talloc context" and a "talloc pointer". Any pointer 19returned from talloc() is itself a valid talloc context. This means 20you can do this: 21 22 struct foo *X = talloc(mem_ctx, struct foo); 23 X->name = talloc_strdup(X, "foo"); 24 25and the pointer X->name would be a "child" of the talloc context "X" 26which is itself a child of mem_ctx. So if you do talloc_free(mem_ctx) 27then it is all destroyed, whereas if you do talloc_free(X) then just X 28and X->name are destroyed, and if you do talloc_free(X->name) then 29just the name element of X is destroyed. 30 31If you think about this, then what this effectively gives you is an 32n-ary tree, where you can free any part of the tree with 33talloc_free(). 34 35If you find this confusing, then I suggest you run the testsuite to 36watch talloc in action. You may also like to add your own tests to 37testsuite.c to clarify how some particular situation is handled. 38 39 40Performance 41----------- 42 43All the additional features of talloc() over malloc() do come at a 44price. We have a simple performance test in Samba4 that measures 45talloc() versus malloc() performance, and it seems that talloc() is 46about 10% slower than malloc() on my x86 Debian Linux box. For Samba, 47the great reduction in code complexity that we get by using talloc 48makes this worthwhile, especially as the total overhead of 49talloc/malloc in Samba is already quite small. 50 51 52talloc API 53---------- 54 55The following is a complete guide to the talloc API. Read it all at 56least twice. 57 58 59=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- 60(type *)talloc(const void *context, type); 61 62The talloc() macro is the core of the talloc library. It takes a 63memory context and a type, and returns a pointer to a new area of 64memory of the given type. 65 66The returned pointer is itself a talloc context, so you can use it as 67the context argument to more calls to talloc if you wish. 68 69The returned pointer is a "child" of the supplied context. This means 70that if you talloc_free() the context then the new child disappears as 71well. Alternatively you can free just the child. 72 73The context argument to talloc() can be NULL, in which case a new top 74level context is created. 75 76 77=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- 78void *talloc_size(const void *context, size_t size); 79 80The function talloc_size() should be used when you don't have a 81convenient type to pass to talloc(). Unlike talloc(), it is not type 82safe (as it returns a void *), so you are on your own for type checking. 83 84 85=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- 86int talloc_free(void *ptr); 87 88The talloc_free() function frees a piece of talloc memory, and all its 89children. You can call talloc_free() on any pointer returned by 90talloc(). 91 92The return value of talloc_free() indicates success or failure, with 0 93returned for success and -1 for failure. The only possible failure 94condition is if the pointer had a destructor attached to it and the 95destructor returned -1. See talloc_set_destructor() for details on 96destructors. 97 98If this pointer has an additional parent when talloc_free() is called 99then the memory is not actually released, but instead the most 100recently established parent is destroyed. See talloc_reference() for 101details on establishing additional parents. 102 103For more control on which parent is removed, see talloc_unlink() 104 105talloc_free() operates recursively on its children. 106 107 108=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- 109int talloc_free_children(void *ptr); 110 111The talloc_free_children() walks along the list of all children of a 112talloc context and talloc_free()s only the children, not the context 113itself. 114 115 116=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- 117void *talloc_reference(const void *context, const void *ptr); 118 119The talloc_reference() function makes "context" an additional parent 120of "ptr". 121 122The return value of talloc_reference() is always the original pointer 123"ptr", unless talloc ran out of memory in creating the reference in 124which case it will return NULL (each additional reference consumes 125around 48 bytes of memory on intel x86 platforms). 126 127If "ptr" is NULL, then the function is a no-op, and simply returns NULL. 128 129After creating a reference you can free it in one of the following 130ways: 131 132 - you can talloc_free() any parent of the original pointer. That 133 will reduce the number of parents of this pointer by 1, and will 134 cause this pointer to be freed if it runs out of parents. 135 136 - you can talloc_free() the pointer itself. That will destroy the 137 most recently established parent to the pointer and leave the 138 pointer as a child of its current parent. 139 140For more control on which parent to remove, see talloc_unlink() 141 142 143=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- 144int talloc_unlink(const void *context, const void *ptr); 145 146The talloc_unlink() function removes a specific parent from ptr. The 147context passed must either be a context used in talloc_reference() 148with this pointer, or must be a direct parent of ptr. 149 150Note that if the parent has already been removed using talloc_free() 151then this function will fail and will return -1. Likewise, if "ptr" 152is NULL, then the function will make no modifications and return -1. 153 154Usually you can just use talloc_free() instead of talloc_unlink(), but 155sometimes it is useful to have the additional control on which parent 156is removed. 157 158 159=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- 160void talloc_set_destructor(const void *ptr, int (*destructor)(void *)); 161 162The function talloc_set_destructor() sets the "destructor" for the 163pointer "ptr". A destructor is a function that is called when the 164memory used by a pointer is about to be released. The destructor 165receives the pointer as an argument, and should return 0 for success 166and -1 for failure. 167 168The destructor can do anything it wants to, including freeing other 169pieces of memory. A common use for destructors is to clean up 170operating system resources (such as open file descriptors) contained 171in the structure the destructor is placed on. 172 173You can only place one destructor on a pointer. If you need more than 174one destructor then you can create a zero-length child of the pointer 175and place an additional destructor on that. 176 177To remove a destructor call talloc_set_destructor() with NULL for the 178destructor. 179 180If your destructor attempts to talloc_free() the pointer that it is 181the destructor for then talloc_free() will return -1 and the free will 182be ignored. This would be a pointless operation anyway, as the 183destructor is only called when the memory is just about to go away. 184 185 186=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- 187void talloc_increase_ref_count(const void *ptr); 188 189The talloc_increase_ref_count(ptr) function is exactly equivalent to: 190 191 talloc_reference(NULL, ptr); 192 193You can use either syntax, depending on which you think is clearer in 194your code. 195 196 197=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- 198void talloc_set_name(const void *ptr, const char *fmt, ...); 199 200Each talloc pointer has a "name". The name is used principally for 201debugging purposes, although it is also possible to set and get the 202name on a pointer in as a way of "marking" pointers in your code. 203 204The main use for names on pointer is for "talloc reports". See 205talloc_report() and talloc_report_full() for details. Also see 206talloc_enable_leak_report() and talloc_enable_leak_report_full(). 207 208The talloc_set_name() function allocates memory as a child of the 209pointer. It is logically equivalent to: 210 talloc_set_name_const(ptr, talloc_asprintf(ptr, fmt, ...)); 211 212Note that multiple calls to talloc_set_name() will allocate more 213memory without releasing the name. All of the memory is released when 214the ptr is freed using talloc_free(). 215 216 217=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- 218void talloc_set_name_const(const void *ptr, const char *name); 219 220The function talloc_set_name_const() is just like talloc_set_name(), 221but it takes a string constant, and is much faster. It is extensively 222used by the "auto naming" macros, such as talloc_p(). 223 224This function does not allocate any memory. It just copies the 225supplied pointer into the internal representation of the talloc 226ptr. This means you must not pass a name pointer to memory that will 227disappear before the ptr is freed with talloc_free(). 228 229 230=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- 231void *talloc_named(const void *context, size_t size, const char *fmt, ...); 232 233The talloc_named() function creates a named talloc pointer. It is 234equivalent to: 235 236 ptr = talloc_size(context, size); 237 talloc_set_name(ptr, fmt, ....); 238 239 240=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- 241void *talloc_named_const(const void *context, size_t size, const char *name); 242 243This is equivalent to: 244 245 ptr = talloc_size(context, size); 246 talloc_set_name_const(ptr, name); 247 248 249=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- 250const char *talloc_get_name(const void *ptr); 251 252This returns the current name for the given talloc pointer. See 253talloc_set_name() for details. 254 255 256=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- 257void *talloc_init(const char *fmt, ...); 258 259This function creates a zero length named talloc context as a top 260level context. It is equivalent to: 261 262 talloc_named(NULL, 0, fmt, ...); 263 264 265=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- 266void *talloc_new(void *ctx); 267 268This is a utility macro that creates a new memory context hanging 269off an exiting context, automatically naming it "talloc_new: __location__" 270where __location__ is the source line it is called from. It is 271particularly useful for creating a new temporary working context. 272 273 274=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- 275(type *)talloc_realloc(const void *context, void *ptr, type, count); 276 277The talloc_realloc() macro changes the size of a talloc 278pointer. The "count" argument is the number of elements of type "type" 279that you want the resulting pointer to hold. 280 281talloc_realloc() has the following equivalences: 282 283 talloc_realloc(context, NULL, type, 1) ==> talloc(context, type); 284 talloc_realloc(context, NULL, type, N) ==> talloc_array(context, type, N); 285 talloc_realloc(context, ptr, type, 0) ==> talloc_free(ptr); 286 287The "context" argument is only used if "ptr" is not NULL, otherwise it 288is ignored. 289 290talloc_realloc() returns the new pointer, or NULL on failure. The call 291will fail either due to a lack of memory, or because the pointer has 292more than one parent (see talloc_reference()). 293 294 295=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- 296void *talloc_realloc_size(const void *context, void *ptr, size_t size); 297 298the talloc_realloc_size() function is useful when the type is not 299known so the typesafe talloc_realloc() cannot be used. 300 301 302=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- 303void *talloc_steal(const void *new_ctx, const void *ptr); 304 305The talloc_steal() function changes the parent context of a talloc 306pointer. It is typically used when the context that the pointer is 307currently a child of is going to be freed and you wish to keep the 308memory for a longer time. 309 310The talloc_steal() function returns the pointer that you pass it. It 311does not have any failure modes. 312 313NOTE: It is possible to produce loops in the parent/child relationship 314if you are not careful with talloc_steal(). No guarantees are provided 315as to your sanity or the safety of your data if you do this. 316 317talloc_steal (new_ctx, NULL) will return NULL with no sideeffects. 318 319=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- 320off_t talloc_total_size(const void *ptr); 321 322The talloc_total_size() function returns the total size in bytes used 323by this pointer and all child pointers. Mostly useful for debugging. 324 325Passing NULL is allowed, but it will only give a meaningful result if 326talloc_enable_leak_report() or talloc_enable_leak_report_full() has 327been called. 328 329 330=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- 331off_t talloc_total_blocks(const void *ptr); 332 333The talloc_total_blocks() function returns the total memory block 334count used by this pointer and all child pointers. Mostly useful for 335debugging. 336 337Passing NULL is allowed, but it will only give a meaningful result if 338talloc_enable_leak_report() or talloc_enable_leak_report_full() has 339been called. 340 341 342=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- 343void talloc_report(const void *ptr, FILE *f); 344 345The talloc_report() function prints a summary report of all memory 346used by ptr. One line of report is printed for each immediate child of 347ptr, showing the total memory and number of blocks used by that child. 348 349You can pass NULL for the pointer, in which case a report is printed 350for the top level memory context, but only if 351talloc_enable_leak_report() or talloc_enable_leak_report_full() has 352been called. 353 354 355=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- 356void talloc_report_full(const void *ptr, FILE *f); 357 358This provides a more detailed report than talloc_report(). It will 359recursively print the ensire tree of memory referenced by the 360pointer. References in the tree are shown by giving the name of the 361pointer that is referenced. 362 363You can pass NULL for the pointer, in which case a report is printed 364for the top level memory context, but only if 365talloc_enable_leak_report() or talloc_enable_leak_report_full() has 366been called. 367 368 369=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- 370void talloc_enable_leak_report(void); 371 372This enables calling of talloc_report(NULL, stderr) when the program 373exits. In Samba4 this is enabled by using the --leak-report command 374line option. 375 376For it to be useful, this function must be called before any other 377talloc function as it establishes a "null context" that acts as the 378top of the tree. If you don't call this function first then passing 379NULL to talloc_report() or talloc_report_full() won't give you the 380full tree printout. 381 382Here is a typical talloc report: 383 384talloc report on 'null_context' (total 267 bytes in 15 blocks) 385 libcli/auth/spnego_parse.c:55 contains 31 bytes in 2 blocks 386 libcli/auth/spnego_parse.c:55 contains 31 bytes in 2 blocks 387 iconv(UTF8,CP850) contains 42 bytes in 2 blocks 388 libcli/auth/spnego_parse.c:55 contains 31 bytes in 2 blocks 389 iconv(CP850,UTF8) contains 42 bytes in 2 blocks 390 iconv(UTF8,UTF-16LE) contains 45 bytes in 2 blocks 391 iconv(UTF-16LE,UTF8) contains 45 bytes in 2 blocks 392 393 394=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- 395void talloc_enable_leak_report_full(void); 396 397This enables calling of talloc_report_full(NULL, stderr) when the 398program exits. In Samba4 this is enabled by using the 399--leak-report-full command line option. 400 401For it to be useful, this function must be called before any other 402talloc function as it establishes a "null context" that acts as the 403top of the tree. If you don't call this function first then passing 404NULL to talloc_report() or talloc_report_full() won't give you the 405full tree printout. 406 407Here is a typical full report: 408 409full talloc report on 'root' (total 18 bytes in 8 blocks) 410 p1 contains 18 bytes in 7 blocks (ref 0) 411 r1 contains 13 bytes in 2 blocks (ref 0) 412 reference to: p2 413 p2 contains 1 bytes in 1 blocks (ref 1) 414 x3 contains 1 bytes in 1 blocks (ref 0) 415 x2 contains 1 bytes in 1 blocks (ref 0) 416 x1 contains 1 bytes in 1 blocks (ref 0) 417 418 419=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- 420void talloc_enable_null_tracking(void); 421 422This enables tracking of the NULL memory context without enabling leak 423reporting on exit. Useful for when you want to do your own leak 424reporting call via talloc_report_null_full(); 425 426 427=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- 428(type *)talloc_zero(const void *ctx, type); 429 430The talloc_zero() macro is equivalent to: 431 432 ptr = talloc(ctx, type); 433 if (ptr) memset(ptr, 0, sizeof(type)); 434 435 436=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- 437void *talloc_zero_size(const void *ctx, size_t size) 438 439The talloc_zero_size() function is useful when you don't have a known type 440 441 442=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- 443void *talloc_memdup(const void *ctx, const void *p, size_t size); 444 445The talloc_memdup() function is equivalent to: 446 447 ptr = talloc_size(ctx, size); 448 if (ptr) memcpy(ptr, p, size); 449 450 451=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- 452char *talloc_strdup(const void *ctx, const char *p); 453 454The talloc_strdup() function is equivalent to: 455 456 ptr = talloc_size(ctx, strlen(p)+1); 457 if (ptr) memcpy(ptr, p, strlen(p)+1); 458 459This functions sets the name of the new pointer to the passed 460string. This is equivalent to: 461 talloc_set_name_const(ptr, ptr) 462 463=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- 464char *talloc_strndup(const void *t, const char *p, size_t n); 465 466The talloc_strndup() function is the talloc equivalent of the C 467library function strndup() 468 469This functions sets the name of the new pointer to the passed 470string. This is equivalent to: 471 talloc_set_name_const(ptr, ptr) 472 473 474=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- 475char *talloc_vasprintf(const void *t, const char *fmt, va_list ap); 476 477The talloc_vasprintf() function is the talloc equivalent of the C 478library function vasprintf() 479 480 481=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- 482char *talloc_asprintf(const void *t, const char *fmt, ...); 483 484The talloc_asprintf() function is the talloc equivalent of the C 485library function asprintf() 486 487This functions sets the name of the new pointer to the passed 488string. This is equivalent to: 489 talloc_set_name_const(ptr, ptr) 490 491 492=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- 493char *talloc_asprintf_append(char *s, const char *fmt, ...); 494 495The talloc_asprintf_append() function appends the given formatted 496string to the given string. 497 498 499=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- 500(type *)talloc_array(const void *ctx, type, uint_t count); 501 502The talloc_array() macro is equivalent to: 503 504 (type *)talloc_size(ctx, sizeof(type) * count); 505 506except that it provides integer overflow protection for the multiply, 507returning NULL if the multiply overflows. 508 509 510=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- 511void *talloc_array_size(const void *ctx, size_t size, uint_t count); 512 513The talloc_array_size() function is useful when the type is not 514known. It operates in the same way as talloc_array(), but takes a size 515instead of a type. 516 517 518=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- 519void *talloc_realloc_fn(const void *ctx, void *ptr, size_t size); 520 521This is a non-macro version of talloc_realloc(), which is useful 522as libraries sometimes want a ralloc function pointer. A realloc() 523implementation encapsulates the functionality of malloc(), free() and 524realloc() in one call, which is why it is useful to be able to pass 525around a single function pointer. 526 527 528=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- 529void *talloc_autofree_context(void); 530 531This is a handy utility function that returns a talloc context 532which will be automatically freed on program exit. This can be used 533to reduce the noise in memory leak reports. 534 535 536=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- 537void *talloc_check_name(const void *ptr, const char *name); 538 539This function checks if a pointer has the specified name. If it does 540then the pointer is returned. It it doesn't then NULL is returned. 541 542 543=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- 544(type *)talloc_get_type(const void *ptr, type); 545 546This macro allows you to do type checking on talloc pointers. It is 547particularly useful for void* private pointers. It is equivalent to 548this: 549 550 (type *)talloc_check_name(ptr, #type) 551 552 553=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- 554talloc_set_type(const void *ptr, type); 555 556This macro allows you to force the name of a pointer to be a 557particular type. This can be used in conjunction with 558talloc_get_type() to do type checking on void* pointers. 559 560It is equivalent to this: 561 talloc_set_name_const(ptr, #type) 562 563=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- 564talloc_get_size(const void *ctx); 565 566This function lets you know the amount of memory alloced so far by 567this context. It does NOT account for subcontext memory. 568This can be used to calculate the size of an array. 569 570