aboutsummaryrefslogtreecommitdiffstats
path: root/uts/common/fs/zfs/dmu_object.c
Commit message (Collapse)AuthorAgeFilesLines
* 10452 ZoL: merge in large dnode feature fixesAndriy Gapon2019-10-151-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | illumos/illumos-gate@946342a260bbae359b48bf142ec1fe40792ee862 https://github.com/illumos/illumos-gate/commit/946342a260bbae359b48bf142ec1fe40792ee862 https://www.illumos.org/issues/10452 illumos is missing a few small follow up ZoL bug fixes for the large dnode feature. We should pull those in. Those commits are in the ZoL tree as (newest to oldest): PR 8435 - 75d6b7ddca269542279975f716a343bb40a79baf - Add missing copyright notice to large_dnode tests PR 7433 - e14a32b1c844d924b9f093375c0badcf10f61741 - Fix object reclaim when using large dnodes PR 6616 - 48fbb9ddbf2281911560dfbc2821aa8b74127315 - Free objects when receiving full stream as clone PR 6695 - 39f56627ae988d09b4e3803c01c22b2026b2310e - receive_freeobjects() skips freeing some object Portions contributed by: Ned Bass <bass6@llnl.gov> Portions contributed by: Tom Caputi <tcaputi@datto.com> Author: Fabian Grünbichler <f.gruenbichler@proxmox.com> Notes: svn path=/vendor-sys/illumos/dist/; revision=353551
* 10406 large_dnode changes broke zfs recv of legacy streamAndriy Gapon2019-08-151-0/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | illumos/illumos-gate@811964cd9f1fbae0fc3b93d116269e9b1fca090a https://github.com/illumos/illumos-gate/commit/811964cd9f1fbae0fc3b93d116269e9b1fca090a https://www.illumos.org/issues/10406 The large dnode changes from 8423 caused problems in zfs recv for a legacy stream. This manifests when attempting to mount the received stream, but the problem is in the receive code. We missed the following commit from ZoL which fixes this. commit da2feb42fb5c7a8c1e1cc67f7a880da9d8e97bc2 Author: Tom Caputi <tcaputi@datto.com> Date: Thu Jun 28 17:55:11 2018 -0400 Fix 'zfs recv' of non large_dnode send streams Currently, there is a bug where older send streams without the DMU_BACKUP_FEATURE_LARGE_DNODE flag are not handled correctly. The code in receive_object() fails to handle cases where drro->drr_dn_slots is set to 0, which is always the case when the sending code does not support this feature flag. This patch fixes the issue by ensuring that that a value of 0 is treated as DNODE_MIN_SLOTS. Author: Tom Caputi <tcaputi@datto.com> Notes: svn path=/vendor-sys/illumos/dist/; revision=351075
* 8423 8199 7432 Implement large_dnode pool featureAndriy Gapon2019-08-121-58/+239
| | | | | | | | | | | | | | | | | | | | | | | | | 8423 Implement large_dnode pool feature 8199 multi-threaded dmu_object_alloc() 7432 Large dnode pool feature llumos/illumos-gate@54811da5ac6b517992fdc173df5d605e4e61fdc0 https://github.com/illumos/illumos-gate/commit/54811da5ac6b517992fdc173df5d605e4e61fdc0 https://www.illumos.org/issues/8423 https://www.illumos.org/issues/8199 https://www.illumos.org/issues/7432 ZoL issues: Improved dnode allocation #6564 Clean up large dnode code #6262 Fix dnode_hold() freeing dnode behavior #8172 Fix dnode allocation race #6414, #6439 Partial: Raw sends must be able to decrease nlevels #6821, #6864 Remove unnecessary txg syncs from receive_object() Closes #7197 Author: Toomas Soome <tsoome@me.com> Notes: svn path=/vendor-sys/illumos/dist/; revision=350898
* 9438 Holes can lose birth time info if a block has a mix of birth timesAlexander Motin2018-08-021-0/+4
| | | | | | | | | | | | | | | | | | | | Ultimately, the problem here is that when you truncate and write a file in the same transaction group, the dbuf for the indirect block will be zeroed out to deal with the truncation, and then written for the write. During this process, we will lose hole birth time information for any holes in the range. In the case where a dnode is being freed, we need to determine whether the block should be converted to a higher-level hole in the zio pipeline, and if so do it when the dnode is being synced out. illumos/illumos-gate@738e2a3ce3b2579222d6855e7fe75b5bcfcddf8d Reviewed by: Matt Ahrens <matt@delphix.com> Reviewed by: George Wilson <george.wilson@delphix.com> Approved by: Robert Mustacchi <rm@joyent.com> Author: Paul Dagnelie <pcd@delphix.com> Notes: svn path=/vendor-sys/illumos/dist/; revision=337200
* 9442 decrease indirect block size of spacemapsAlexander Motin2018-08-021-2/+12
| | | | | | | | | | | | | | | | | | Updates to indirect blocks of spacemaps can contribute significantly to write inflation. Therefore we want to reduce the indirect block size of spacemaps from 128K to 16K. illumos/illumos-gate@221813c13b43ef48330b03725e00edee85108cf1 Reviewed by: Serapheim Dimitropoulos <serapheim.dimitro@delphix.com> Reviewed by: George Wilson <george.wilson@delphix.com> Reviewed by: Albert Lee <trisk@forkgnu.org> Reviewed by: Igor Kozhukhov <igor@dilos.org> Approved by: Dan McDonald <danmcd@joyent.com> Author: Matthew Ahrens <mahrens@delphix.com> Notes: svn path=/vendor-sys/illumos/dist/; revision=337167
* 9328 zap code can take advantage of c99Alexander Motin2018-08-011-3/+9
| | | | | | | | | | | | | | | | 9329 panic in zap_leaf_lookup() due to concurrent zapification illumos/illumos-gate@bf26014c5541b6119f34e0d95294b7f2eb105ac2 Reviewed by: Steve Gonczi <steve.gonczi@delphix.com> Reviewed by: George Wilson <george.wilson@delphix.com> Reviewed by: Pavel Zakharov <pavel.zakharov@delphix.com> Reviewed by: Brad Lewis <brad.lewis@delphix.com> Approved by: Dan McDonald <danmcd@joyent.com> Author: Matthew Ahrens <mahrens@delphix.com> Notes: svn path=/vendor-sys/illumos/dist/; revision=337027
* 7801 add more by-dnode routinesAndriy Gapon2017-04-141-4/+5
| | | | | | | | | | | | | | | | | | | | | | | | illumos/illumos-gate@b0c42cd4706ba01ce158bd2bb1004f7e59eca5fe https://github.com/illumos/illumos-gate/commit/b0c42cd4706ba01ce158bd2bb1004f7e59eca5fe https://www.illumos.org/issues/7801 Add *_by_dnode() routines for accessing objects given their dnode_t *, this is more efficient than accessing the object by (objset_t *, uint64_t object). This change converts some but not all of the existing consumers. As performance-sensitive code paths are discovered they should be converted to use these routines. Ported from: https://github.com/zfsonlinux/zfs/commit/ 0eef1bde31d67091d3deed23fe2394f5a8bf2276 Reviewed by: Matthew Ahrens <mahrens@delphix.com> Reviewed by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed by: Pavel Zakharov <pavel.zakharov@delphix.com> Approved by: Robert Mustacchi <rm@joyent.com> Author: bzzz77 <bzzz.tomas@gmail.com> Notes: svn path=/vendor-sys/illumos/dist/; revision=316914
* 7430 Backfill metadnode more intelligentlyAndriy Gapon2017-04-141-11/+20
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | illumos/illumos-gate@af346df58864e8fe897b1ff1a3a4c12f9294391b https://github.com/illumos/illumos-gate/commit/af346df58864e8fe897b1ff1a3a4c12f9294391b https://www.illumos.org/issues/7430 Description and patch from brought over from the following ZoL commit: https:// github.com/zfsonlinux/zfs/commit/68cbd56e182ab949f58d004778d463aeb3f595c6 Only attempt to backfill lower metadnode object numbers if at least 4096 objects have been freed since the last rescan, and at most once per transaction group. This avoids a pathology in dmu_object_alloc() that caused O(N^2) behavior for create-heavy workloads and substantially improves object creation rates. As summarized by @mahrens in #4636: "Normally, the object allocator simply checks to see if the next object is available. The slow calls happened when dmu_object_alloc() checks to see if it can backfill lower object numbers. This happens every time we move on to a new L1 indirect block (i.e. every 32 * 128 = 4096 objects). When re-checking lower object numbers, we use the on-disk fill count (blkptr_t:blk_fill) to quickly skip over indirect blocks that don’t have enough free dnodes (defined as an L2 with at least 393,216 of 524,288 dnodes free). Therefore, we may find that a block of dnodes has a low (or zero) fill count, and yet we can’t allocate any of its dnodes, because they've been allocated in memory but not yet written to disk. In this case we have to hold each of the dnodes and then notice that it has been allocated in memory. The end result is that allocating N objects in the same TXG can require CPU usage proportional to N^2." Add a tunable dmu_rescan_dnode_threshold to define the number of objects that must be freed before a rescan is performed. Don't bother to export this as a module option because testing doesn't show a compelling reason to change it. The vast majority of the performance gain comes from limit the rescan to at most once per TXG. Reviewed by: Alek Pinchuk <alek@nexenta.com> Reviewed by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed by: Matthew Ahrens <mahrens@delphix.com> Approved by: Gordon Ross <gordon.w.ross@gmail.com> Author: Ned Bass <bass6@llnl.gov> Notes: svn path=/vendor-sys/illumos/dist/; revision=316868
* 6370 ZFS send fails to transmit some holesAlexander Motin2016-03-101-1/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Reviewed by: Matthew Ahrens <mahrens@delphix.com> Reviewed by: Chris Williamson <chris.williamson@delphix.com> Reviewed by: Stefan Ring <stefanrin@gmail.com> Reviewed by: Steven Burgess <sburgess@datto.com> Reviewed by: Arne Jansen <sensille@gmx.net> Approved by: Robert Mustacchi <rm@joyent.com> Author: Paul Dagnelie <pcd@delphix.com> In certain circumstances, "zfs send -i" (incremental send) can produce a stream which will result in incorrect sparse file contents on the target. The problem manifests as regions of the received file that should be sparse (and read a zero-filled) actually contain data from a file that was deleted (and which happened to share this file's object ID). Note: this can happen only with filesystems (not zvols, because they do not free (and thus can not reuse) object IDs). Note: This can happen only if, since the incremental source (FromSnap), a file was deleted and then another file was created, and the new file is sparse (i.e. has areas that were never written to and should be implicitly zero-filled). We suspect that this was introduced by 4370 (applies only if hole_birth feature is enabled), and made worse by 5243 (applies if hole_birth feature is disabled, and we never send any holes). The bug is caused by the hole birth feature. When an object is deleted and replaced, all the holes in the object have birth time zero. However, zfs send cannot tell that the holes are new since the file was replaced, so it doesn't send them in an incremental. As a result, you can end up with invalid data when you receive incremental send streams. As a short-term fix, we can always send holes with birth time 0 (unless it's a zvol or a dataset where we can guarantee that no objects have been reused). Closes #37 Notes: svn path=/vendor-sys/illumos/dist/; revision=296609
* 5960 zfs recv should prefetch indirect blocksAlexander Motin2015-08-121-0/+5
| | | | | | | | | | | | | | | | 5925 zfs receive -o origin= Reviewed by: Prakash Surya <prakash.surya@delphix.com> Reviewed by: Matthew Ahrens <mahrens@delphix.com> Author: Paul Dagnelie <pcd@delphix.com> While running 'zfs recv' we noticed that every 128th 8K block required a read. We were seeing that restore_write() was calling dmu_tx_hold_write() and the indirect block was not cached. We should prefetch upcoming indirect blocks to avoid having to go to disk and blocking the restore_write(). Notes: svn path=/vendor-sys/illumos/dist/; revision=286704
* 3693 restore_object uses at least two transactions to restore an objectXin LI2014-10-091-39/+3
| | | | | | | | | | | | | Reviewed by: Christopher Siden <christopher.siden@delphix.com> Reviewed by: George Wilson <george.wilson@delphix.com> Reviewed by: Andriy Gapon <andriy.gapon@hybridcluster.com> Approved by: Robert Mustacchi <rm@joyent.com> Author: Matthew Ahrens <mahrens@delphix.com> illumos/illumos-gate@e77d42eaa49fe55bfae1e0e0065c6e99affc001b Notes: svn path=/vendor-sys/illumos/dist/; revision=272804
* 4171 clean up spa_feature_*() interfacesAndriy Gapon2013-11-201-0/+53
| | | | | | | | | | 4172 implement extensible_dataset feature for use by other zpool features illumos/illumos-gate@2acef22db7808606888f8f92715629ff3ba555b9 Notes: svn path=/vendor-sys/illumos/dist/; revision=258374
* Update vendor/illumos/dist and vendor-sys/illumos/distMartin Matuska2013-03-121-2/+3
| | | | | | | | | | to illumos-gate 13980:d7059eb1884c Illumos ZFS issues: 3598 want to dtrace when errors are generated in zfs Notes: svn path=/vendor-sys/illumos/dist/; revision=248217
* Add kernel part (uts) with ZFS to vendor/illumosMartin Matuska2012-07-011-0/+196
illumos-gate revision 13742:b6bbdd77139c Notes: svn path=/vendor/illumos/dist/; revision=237927
generated by cgit v1.2.3 (git 2.25.1) at 2022-07-01 11:54:06 +0000