Inpainting is a well-known technique in the context of image and art restoration, where paint losses are filled up to the level of the surrounding paint and then coloured to match. Analogue tasks can be found in 3D geometry processing, as digital representations of real-world objects often contain holes, due to hindrances during data acquisition or as a consequence of interactive modelling operations. We present a novel approach to automatically fill-in holes in structured surfaces where smooth hole filling is not sufficient. Previous approaches inspired by texture synthesis algorithms require specific spatial structures to identify holes and possible candidate fragments to be copied to defective regions. Consequently, the results depend heavily on the choice and location of these auxiliary structures, such that for instance symmetries are not reconstructed faithfully. In contrast, our approach is based on local neighbourhoods and therefore insensitive with respect to similarity transformations. We use socalled guidance surfaces to guide and prioritise the atomic filling operations, such that even non-trivial and larger holes can be filled consistently. The guidance surfaces are automatically computed and iteratively updated during the filling process, but can also incorporate any additional information about the surface, if available.