Lines Matching refs:metadata
16 Almost all metadata on XFS is dynamically allocated. The only fixed location
17 metadata is the allocation group headers (SB, AGF, AGFL and AGI), while all
18 other metadata structures need to be discovered by walking the filesystem
31 However, if we scale the filesystem up to 1PB, we now have 10x as much metadata
42 One of the problems with the current metadata format is that apart from the
43 magic number in the metadata block, we have no other way of identifying what it
45 you can't look at a single metadata block in isolation and say "yes, it is
49 verification of metadata values, looking for values that are in range (and hence
56 Hence we need to record more information into the metadata to allow us to
57 quickly determine if the metadata is intact and can be ignored for the purpose
60 self describing metadata.
62 The first, fundamental requirement of self describing metadata is that the
63 metadata object contains some form of unique identifier in a well known
65 hence parse and verify the metadata object. IF we can't independently identify
66 the type of metadata in the object, then the metadata doesn't describe itself
69 Luckily, almost all XFS metadata has magic numbers embedded already - only the
73 numbers in the metadata objects. That is, if it has the current magic number,
74 the metadata isn't self identifying. If it contains a new magic number, it is
76 metadata object at runtime, during forensic analysis or repair.
78 As a primary concern, self describing metadata needs some form of overall
79 integrity checking. We cannot trust the metadata if we cannot verify that it has
81 integrity check, and this is done by adding CRC32c validation to the metadata
82 block. If we can verify the block contains the metadata it was intended to
85 CRC32c was selected as metadata cannot be more than 64k in length in XFS and
87 metadata blocks. CRC32c is also now hardware accelerated on common CPUs so it is
95 Self describing metadata needs to contain enough information so that the
96 metadata block can be verified as being in the correct place without needing to
97 look at any other metadata. This means it needs to contain location information.
98 Just adding a block number to the metadata is not sufficient to protect against
103 Another key information point in forensic analysis is knowing who the metadata
106 of the block is important as it allows us to find other related metadata to
111 in the metadata we have no idea of the scope of the corruption. If we have an
112 owner field in the metadata object, we can immediately do top down validation to
115 Different types of metadata have different owner identifiers. For example,
118 contents of the owner field are determined by the type of metadata object we are
122 Self describing metadata also needs to contain some indication of when it was
125 metadata blocks based on modification times is important as it can indicate
130 For example, we can determine whether a metadata object is supposed to be free
133 compared to when the metadata object itself was last written. If the free space
137 To provide this "written timestamp", each metadata block gets the Log Sequence
141 the LSN we can tell if the corrupted metadata all belonged to the same log
143 the first and last instance of corrupt metadata on disk and, further, how much
150 Validation of self-describing metadata takes place at runtime in two places:
156 modification process, and seeks only to check that the metadata is what it says
157 it is and that the metadata fields are within bounds and internally consistent.
160 metadata, or there may be corruption of interblock relationships (e.g. corrupted
165 For read verification, the caller needs to specify the expected type of metadata
166 that it should see, and the IO completion process verifies that the metadata
178 object specific metadata validation. If any of these checks fail, then the
202 Depending on the metadata, this information may be part of a header structure
203 separate to the metadata contents, or may be distributed through an existing
204 structure. The latter occurs with metadata that already contains some of this
207 Other metadata may have different formats for the information, but the same
217 well. hence the additional metadata headers change the overall format
218 of the metadata.
323 This will verify the internal structure of the metadata before we go any
324 further, detecting corruptions that have occurred as the metadata has been
325 modified in memory. If the metadata verifies OK, and CRCs are enabled, we then
327 metadata. Once this is done, we can issue the IO.