Figure 6 shows two paintings by Vermeer and their weave maps: The Girl Interrupted at Her Music from the Frick Collection and Young Woman with a Water Pitcher from the Metropolitan Museum of Art. The weave maps show that these two paintings were made from the same bolt of fabric.

Hunting for Weave Matches: Computation in Art Scholarship

Abstract

Manual thread counts are widely used in the technical analysis of a painting’s canvas (Franken 2017). When the average thread counts of two canvases are similar, this suggests that the two canvases are derived from the same bolt; conversely, if they differ substantially, they cannot have originated from the same roll. Yet average thread counts alone, no matter how close, are not enough to establish that two canvases were manufactured together. With the advent of computer methods, it is now possible to “count” the threads at every location in a canvas. Transforming this numerical procedure into an image produces weave maps such as in Figure 1, which display values of the thread count as colored dots. Striped patterns in the colors are local thread count variations attributable to the mechanics of the weaving process. When two canvases have similar patterns of stripes, such as the two by Vermeer in Figure 1, they can be presumed to have a common origin, thus placing the canvases (and the paintings) in the same place at the same time. This paper describes an organized procedure for carrying out the search for weave matches, includes a summary of the findings of the Counting Vermeer project, and introduces a new match between two of Johannes Vermeer’s canvases.

Johnson & Sethares (Figure 1)

Figure 1: Johannes Vermeer, The Geographer (top), ca. 1668–1669, Oil on canvas, 53 x 46.6 cm. Städelsches Kunstinstitut und Städtische Galerien, Frankfurt am Main and Johannes Vermeer, The Astronomer (bottom, rotated 180◦), ca. 1668, Oil on canvas, 50 x 45 cm. Musée du Louvre, Paris (artwork from Wikimedia Commons), and corresponding vertical weave maps (at right). Each pixel in the weave map represents the value of a thread count at a single evaluation point, which is carried out automatically over the complete canvas. To convert the similarity assessment into a visual task, the density values for each evaluation tile are transformed into colored squares that indicate the densities of the vertical threads. The immediately obvious defining feature in these two maps is the pattern of stripes. As this pattern is the result of the manufacturing process, it will be shared by bolt mates. This match was first presented in “Canvas Weave Match Supports Designation of Vermeer’s Geographer and Astronomer as a Pendant Pair” (Johnson and Sethares 2016).

Background

In March 2006, Ella Hendricks, head of painting conservation at the Van Gogh Museum, agreed to tutor Rick Johnson (the first author of this article) in the activities of a conservator with the goal of expanding the use of computer-based tools in painting analysis. At the time, thread counting was typically accomplished using x-radiograph film of a canvas posted on a light box and measured with a magnifying headset. After shadowing the department for fifteen months, the author recognized that thread counting could serve as a candidate for automated analysis via Fourier methods. Fourier analysis is a common tool taught to undergraduate electrical engineers that is useful for estimating the periodic components in a signal. In the thread counting application, the “signal” is the greyscale intensity fluctuations visible in the x-radiograph corresponding to the threads. Once the potential of Fourier analysis to automate canvas thread counting was confirmed, the Thread Count Automation Project (TCAP) was formed in 2007 to coordinate teams of experts in digital signal processing. The teams tested a variety of procedures and a variety of ways of presenting the results. One fruitful method visualizes the thread count data over a complete canvas using color to represent the varying thread densities. The resulting weave maps (such as in Figure 1) are often striped, and offer a detailed view of the small variations in the density pattern of the weave. These variations often extend over the width and breadth of the roll originally containing the canvas. A weave match occurs when the weave maps from two different canvases have the same pattern of stripes. This has become compelling forensic evidence in establishing rollmate status between two separate canvases.

When combined with information about a painter’s studio practice, material data (e.g. the range of ground layer materials used by the artist), and documentation (e.g. in letters, financial transactions, and memoirs), canvas weave matches can assist in authentication, dating, and inference of artist’s intent, as Liedtke, Johnson, and Johnson recognized in 2012 for paintings by Vermeer. That same year, document-rich studies incorporating weave matches appeared of paintings by Vincent van Gogh (Tilborgh et al. 2012) and Diego Velázquez (Pérez d’Ors, Johnson, and Johnson 2012). In the case of Vermeer, for whom substantial studio documentation is lacking, the eight weave match pairs discovered so far (Johnson Counting 2017) from among Vermeer’s thirty-four paintings on canvas (Liedtke 2008) potentially offer additional insight into unanswered questions, in particular those regarding dating. When a weave match is made between canvases that stylistic analysis dates as several years apart, this can challenge the prevailing sense that an artist would be unlikely to hoard canvas for a long period. Assembling a weave match study across the entire oeuvre of a painter can provide valuable data regarding chronology. A comparison of weave maps for paintings by different artists from the same period and place could help to establish patterns of interaction among the artists.

In the decade since the creation of the first automated thread counting procedures, researchers have used weave maps of thread density and angle to analyze Old Master European paintings from the fifteenth to the early twentieth centuries. Striped weave maps have also been produced for twelfth and thirteenth-century Chinese silk painting supports. The extension to other fabrics with periodic weave patterns can be envisioned as long as they can be imaged with resolution sufficient for a human to count the threads, this might include clothing, flags, and other woven cultural heritage objects. Thus, this decade-long project to develop procedures that use tools from digital signal processing as aids to art historical analysis of paintings on fabric demonstrates the potentials of technology and the trans-disciplinary collaboration between art historians, signal processing engineers, and the institutions that provide the support and data that enable collaborations. The objective of the present paper is to document the procedure employed in the study of Vermeer’s canvases (Johnson Counting 2017) as it advances from scanned x-radiographs, through weave maps, and onto the hunt for weave matches. The audience for this document is presumed to be the end users of the weave match reports; no knowledge of digital signal processing or computer programming is presumed, and therefore, mathematical language is avoided.

A Procedure for Finding Weave Matches

Hunting for Weave Matches presents the complete procedure for comparing possible weave matches and attempting to locate matched pairs. The procedure is accomplished in several stages. First, it is necessary to gather appropriate x-radiographs and transform them into digitized form. Second, manual thread counting is needed both to provide a ground truth and to initialize the computer methods. Third, computer methods draw the weave maps of the individual canvases. Fourth, the weave maps must be compared to each other, and plausible matches detected. Finally, possible matches must be studied by referring back to the x-ray images, to the paintings, and to the incorporation of art historical knowledge. Though it may not be immediately obvious, many of these steps require interaction between the eye of the conservator (or art historian) and the computational algorithm, between the expert and the machine.

The success of the automated thread counting project provides an example of collaboration between art history and computational science where software tools were developed to target a basic problem of interest to art historians and conservators. The human expert is an inherent part of the decision process and a crucial participant in the application of such tools. In the creation of Counting Vermeer (Johnson 2017), x-radiographs providing full-painting coverage of all thirty-four of Vermeer’s works on canvas were digitized, stitched, hand counted, weave mapped, and compared. In documenting the result of the interactions between art historians and signal processing engineers over the course of the Counting Vermeer project, and by formalizing the procedures developed in this collaboration, this paper hopes to show how such interactions can be mutually beneficial.

Compiling a formalized procedure may appear somewhat pedantic; after all, given software for drawing weave maps, what else needs to be done but to press the “run” button? There are several issues, however. First, how is the basic data to be gathered? For example, if an x-ray is scanned at too low a resolution, the evidence for existing stripes may be ambiguous or lost entirely. Secondly, when conducting the manual count, where and how many locations should be counted? Finally, when comparing weave maps, how should stripes be assessed as aligned and a match declared? By following the procedure outlined in this paper, and diagrammed as a “flow chart” in Figure 2, it is possible to carry out the procedure in a reproducible manner. This does not mean that interpretations of weave maps are indisputable; rather, it demonstrates that the basic data on which arguments are based can be re-derived in a systematic manner by any researcher. Yet the interpretation of that data will always require an experienced eye.

Johnson & Sethares (Figure 2)

Figure 2: Schematic flow chart of the complete “hunting for weave maps” procedure.

The Hunt

Digitizing

A key parameter in the study of digital x-radiographs is the size of each pixel, which is typically specified in terms of dots per inch and abbreviated dpi.

Step 1: For all canvases to be studied and compared, obtain x-radiographs digitized (scanned) at sufficiently high resolution. Ten dots (pixels) per thread is the recommended resolution.

For 10 dots per thread, the dpi is 25.4 times the thread count in threads per centimeter (th/cm). For canvas with a thread count of 12 th/cm a dpi of 300 will provide 10 dots per thread. For a thread count of 24 th/cm a resolution of 600 dpi provides 10 dots per thread. Fewer than 10 dots per thread can sometimes provide an adequate image. But fewer than 5 dots per thread typically produces images that are difficult to hand count. A simple rule of thumb is that the chosen resolution needs to be high enough to make manual thread counting feasible.

Stitching

When the canvas is too large to be covered by a single x-radiograph, it is necessary to take multiple images.

Step 2: Construct a composite of the full painting with the constituent digitized x-radiographs using computational stitching.

In order to reassemble these segments, some overlap is necessary (at least 10%) between all adjacent x-radiographs. The re-assembly process can be done in two different ways. The weave maps can be calculated first and then a drawing program (such as Photoshop) can be used to align the stripes in the constituent images, or a stitching algorithm can be used to reconstruct a full x-radiograph, from which the weave map is then calculated (Johnson, Erdmann, and Johnson 2011; Conover, Delaney, and Loew 2013). Since the stitching process is somewhat complicated, there is the possibility that a discontinuity in the stitching may introduce artifacts into the weave maps (for example, a feature may be distorted or may appear doubled). For smaller canvases where a single x-radiograph can cover the complete canvas, skip this step.

Manual Thread Counting

Given the repetitive nature of the thread counting task, it is tempting to delegate all the tedious tasks to the computer. However, there are two good reasons to conduct an initial manual count. The first is that computer-based calculations of “thread count” do not literally count individual threads the way a person might. Rather, they carry out a numerical procedure whose result closely approximates the thread count (Sethares “Automated” 2017). The second reason to conduct manually-assisted thread counts is that all current software methods for thread counting need to be initialized or trained. For example, the Fourier method (Sethares “Automated” 2017) needs to be told “approximately” what the thread count is, and it can then quite reliably fine-tune this answer. Yet if the initial guess the software is given is far from the correct value, then the resulting computed result may well also be incorrect. Thus, a manual thread count is needed to ensure that the computer-generated counts are plausible.

Step 3: For each canvas, perform a manual thread count in several (5 to 15 are suggested) patches (1 to 2 cm squares are suggested) across the painting (with no two counting the same threads as they continue through the painting canvas) of threads in both the horizontal and vertical directions (relative to the painting orientation). Software is available to assist and document this manual thread counting (Sethares “Computer-Assisted” 2017).

Automated Counting

Weave maps compute “thread counts” in square tiles of size x by x centered on a grid of points that are the same distance d apart in the vertical and horizontal directions. The separation should be no larger than the edge length of the square evaluation tiles. For example, it is common to maintain a simple integer relationship x = n d between x and d where n = 1, 2, 3, or 4. Typical choices for x range from 0.5 to 2 cm. Since radiographs are in one to one correspondence with the canvas, these blocks can be visually mapped directly onto the canvas.

Step 4: For a suitable range of values of x (e.g. 0.5, 1.0, 1.5, and 2 cm) and a suitable range of values for d (e.g. 1, 2, 3, and 4) compute the thread counts in tiles covering the composite x-radiograph.

Counting software using Fourier spectral analysis is available (Sethares “Computer-Assisted” 2017). The details of extracting accurate thread counts from peaks in the spectrum depend on the weave type, e.g. simple (also known as tabby) and twill (Johnson, Johnson, and Erdmann 2013). Most Old Master paintings are on simple weave canvas. This spectral method works well in approximating the thread count only if the threads are sufficiently regularly spaced. If the spacing is too irregular and not approximately periodic, the spectral information, while no longer necessarily providing a close match to the actual thread count, can still be used as an image feature allowing weave comparison.

Match Plausibility via Scatter Plot

If the average counts in two paintings of these manual thread counts are too different, i.e. they differ by more than 2 th/cm in both directions, then they cannot be rollmates and continuing to draw weave maps is unnecessary if the only issue to be determined is if they are rollmates. One way to organize the data from a large number of paintings is to draw a scatter plot as in Figure 3. This shows all the computed counts in both directions for all thirty-four Vermeer paintings on canvas, labeled by L number (Liedtke 2008). Canvases that are far apart need not be considered for possible matching. On the other hand, canvases that are nearby in the scatter plot deserve further consideration.

Step 5: Given the computed thread count averages (both horizontal and vertical directions) for each canvas under examination, draw the scatter plot by plotting the smallest value on the horizontal axis and the largest value on the vertical axis.

Johnson & Sethares (Figure 3)

Figure 3: Scatter plot of average thread counts. Each small circle plots the thread count (smaller direction vs. larger direction) and are labelled with the adjacent L numbers. Tight count matches of a specific painting are within a 2 cm square box centered on coordinates of a given painting.

Weave Maps

Step 6: For each painting under examination, map the thread count values into a color image using a colorbar to associate the numbers to colors. Construct one weave map for the horizontal threads and another for the vertical threads.

Software for converting the computed thread counts from each tile into weave maps is available (Sethares “Automated” 2017). The average of the computed thread counts in one thread direction should be used as the center of the colorbar for the corresponding weave map. The range of the colorbar should extend from 3 th/cm below the center value of the color bar to 3 th/cm above it. (These numerical suggestions are for the typical canvas thread counts of fifteenth to early twentieth-century European canvases, which typically range from around 6 th/cm up to the low 20s.) The weave map puts the color for the computed thread count value for each evaluation tile in a d by d sized square centered on the point on which the evaluation tile is centered.

Tiling Selection to Reveal Stripes

If the square evaluation tile side dimension x is too small (which is relative to the degree of count variability across the painting), a weave map will appear random (e.g. speckled, without stripes) and will not be useful for weave matching. If x is too large, the weave map will have a blob-like appearance with no apparent stripes. This is illustrated in Figure 4, which shows a single canvas analyzed with a variety of x and d values.

Step 7: Examine the resulting weave maps for various x and d. Select an x and a d that allow the weave maps of each canvas being counted to exhibit stripes in both of its two thread directions, if possible.

Sometimes the x-radiograph image can be of such poor quality that stripes appear in only one weave map direction— horizontal (H) or vertical (V)—no matter how x and d are chosen. The weave match hunting procedure can continue in such a case but this limits the matching test orientation options.

Johnson & Sethares (Figure 4)

Figure 4: Using different-sized regions (parameter d) examine the canvas at different scales (parameter x) and can lead to different results for thread density maps. Weave maps drawn in various color schemes display the same information, but the stripes may be easier to see in some color schemes.

Re-Coloring the Weave Maps

When comparing any two (or more) weave maps, it is necessary that they be plotted with the same color bars. This is one of the reasons why the weave-map data must be maintained in numerical form (rather than as static jpeg images, for instance).

Step 8: For the x and d chosen in the previous step (that cause stripes to appear in at least one of the two weave maps—with one of horizontal threads and the other of the vertical threads—of each canvas) use the same color bar to compose weave maps drawn to the same scale for all of the paintings to be compared for possible weave matches.

The color bar should be centered on a value that is near either the horizontal (H) or vertical (V) average count of the two canvases. If one canvas is to be compared to multiple canvases with which it has a count match, i.e. in one comparison of H to H and V to V or H to V and V to H the average thread counts computed across the entire images are close to one another, the average H and V thread counts of the one canvas to be compared to several others should be used as color bar centers. Weave maps to the same physical scale should have the same relative sizes as the (composite) x-radiographs of the canvases.

This process is illustrated in the accompanying video demonstration (at about 40 seconds, when the title is “Visually compare L13 with”). Subtle recoloring of the weave maps, originally colored about their individual average counts, occurs when they are collated to compare with L13. The group of paintings chosen for comparison to L13 have average counts within 1 th/cm of each other in both directions. Such “tight” count matches, within 1 th/cm in both directions for a H to H and V to V or H to V and V to H comparison, were chosen because the first six weave matches among Vermeer’s paintings on canvas were found to be within this degree of similarity (Johnson “Exploiting” 2017).

Looking for Matches

Step 9: Compare the horizontal thread stripe patterns of the weave maps of each pair of canvases to see if for some relative positioning the stripe patterns align. Repeat for weave maps of the vertical threads.

These weave maps must use the same x and d, the same procedure for extraction of thread counts from the spectra of the evaluation tiles, and the same color bars. Various orientations should be considered, including one on the right and one on the left of images with horizontal stripes and the reverse left-right pairing. In addition, right-left and left-right orientations should be investigated after rotation by 180 degrees of one of the maps. Furthermore, the right-left and left-right orientations should be considered after one of the weave maps has been right-left flipped. Flipping will be necessary if the ground has been applied to opposite sides of the original canvas.

This procedure is shown in the accompanying video demonstration (0:49–1:26, when comparing L13 with L07, L08, and L09). Slide the images around in all orientations until a match appears, often seeming to suddenly “lock in.” For the vertical thread weave maps of the canvases being compared, above-below and below-above configurations (rather than right-left and left-right configurations) should be examined for vertical stripes, along with the corresponding rotation and flip.

Rotating Weave Maps for Cross-Comparison

Step 10: Repeat the visual matching of Step 9 to compare the horizontal threads weave map of one canvas and the vertical threads weave map for the second canvas rotated by 90 degrees. Similarly, repeat for the vertical threads weave map of one canvas and the horizontal threads weave map for the second canvas rotated by 90 degrees.

Only one (or none) of these four orientation comparisons outlined in Steps #9 and #10, i.e. H to H, V to V, H to V, and V to H, can reveal matching stripe patterns. Stripe alignment is a judgment call to be made by the art expert.

A New Match

The accompanying video demonstration presents the process of looking for matches for L13 (Figure 5). This visual comparison procedure must be repeated for each of the plausible candidates that are nearby in the scatter plot of Figure 3. In the end, the video shows that L09 provides a good match for L13!

Johnson & Sethares (Figure 5)

Figure 5: A screen shot of the accompanying video demonstration shows the comparison procedure, which requires the eye of an expert.

These two weave matching paintings are Girl Interrupted at Her Music, The Frick Collection (L09) and Young Woman with a Water Pitcher, The Metropolitan Museum of Art (L13) (Figure 6). This discovery was welcomed by the staff of The Frick Collection as the painting surface has suffered from pigment degradation and overcleaning prior to its purchase by the institution, a situation that complicates efforts to date the painting. This match was discovered in June 2017 (after the completion of the text for Counting Vermeer (Johnson 2017)) by the process described here and illustrated in the video. If weave matching pairs among Vermeer’s paintings are assumed to imply that the two paintings were painted at around the same time, this may challenge the range of datings by various catalogers of Vermeer’s paintings based solely on stylistic analysis (Janson).

Johnson & Sethares (Figure 6)

Figure 6: Recent weave match of Girl Interrupted at Her Music, The Frick Collection (L09) and Young Woman with a Water Pitcher, The Metropolitan Museum of Art (L13). L09 is rotated 90 degrees to match its vertical thread map with the horizontal thread map of L13, as shown in the accompanying video.

Concluding Remarks

This paper presents an organized procedure for canvas analysis resulting from a decade of intense collaboration between art museum curators, conservators, and signal processing engineers (Johnson “Thread Count”). The discovery and utilization of striped weave maps required an immersion by engineers in the problems and techniques of art experts; a willingness (unusual at the start of the project in 2007) by participating museums to share high resolution scientific data with researchers outside the museum community; visualization of the numerical results in a form suited to the advanced close-looking skills of art experts; the commitment of the computational and art experts to find a common (non-mathematical) cross-disciplinary language necessary to sustain the collaboration and move into uncharted territory; and the immersion of the art experts in the interpretation of the results (rather than a futile attempt to displace experts with computer-generated “knowledge”). The results have far exceeded the preconceptions of all parties involved.

 

Bibliography

Conover, Devin M., John K. Delaney, and Murray H. Loew. 2013. “Automatic Registration and Mosaicking of Conservation Images.” Proceedings of the Society of Photo-Optical Instrumentation Engineers (SPIE) 8790, Optics for Arts, Architecture, and Archaeology IV. 87900A. DOI:10.1117/12.2021318.

Franken, Michiel. 2017. “Sixty Years of Thread Counting.” In Counting Vermeer: Using Weave Maps to Study Vermeer’s Canvases, edited by C. Richard Johnson, Jr. and William A. Sethares. The Hague: RKD-Nederlands Instituut voor Kunstgeschiedenis. http://countingvermeer.rkdmonographs.nl.

Janson, Jonathan. n.d. “Dating Vermeer’s Paintings.” Essential Vermeer. http://www.essentialvermeer.com/references/dates.html#.WXkKSVGQzIU.

Johnson, C. Richard Jr. n.d. “Thread Count Automation Project (TCAP) Timeline (2007–2014).” http://people.ece.cornell.edu/johnson/tcap.html.

———. 2017. “Exploiting Weave Maps.” In Counting Vermeer: Using Weave Maps to Study Vermeer’s Canvases, edited by C. Richard Johnson, Jr. and William A. Sethares. The Hague: RKD-Nederlands Instituut voor Kunstgeschiedenis. http://countingvermeer.rkdmonographs.nl.

Johnson, C. Richard Jr., and William A. Sethares. 2017. “Canvas Weave Match Supports Designation of Vermeer’s Geographer and Astronomer as a Pendant Pair.” Journal of Historians of Netherlandish Art 9, no. 1 (Winter). DOI:  10.5092/jhna.2017.9.1.17.

Johnson, C. Richard Jr., and William A. Sethares, eds. 2017. Counting Vermeer: Using Weave Maps to Study Vermeer’s Canvases. The Hague: RKD-Nederlands Instituut voor Kunstgeschiedenis. http://countingvermeer.rkdmonographs.nl.

Johnson, Don H., Robert G. Erdmann, and C. Richard Johnson, Jr. 2011. “Whole-Painting Canvas Analysis Using High- and Low-Level Features.” Proceedings of the International Conference on Acoustics, Speech, and Signal Processing. DOI: 10.1109/ICASSP.2011.5946567.

Johnson, Don H., C. Richard Johnson, Jr., and Robert G. Erdmann. 2013. “Weave Analysis of Paintings on Canvas from Radiographs.” Signal Processing (Special Issue on Image Processing for Art Investigation) 93 (March): 527–40.

Liedtke, Walter. 2008 Vermeer: The Complete Paintings. Ghent, Belgium: Ludion Press.

Liedtke, Walter, C. Richard Johnson, Jr., and Don H. Johnson. 2012. “Canvas Matches in Vermeer: A Case Study in the Computer Analysis of Fabric Supports.” Metropolitan Museum Journal 47: 99–106.

Pérez d’Ors, Pablo, C. Richard Johnson, Jr., and Don H. Johnson. 2012. “Velázquez in Fraga: a New Hypothesis about the Portraits of El Primo and Philip IV.” The Burlington Magazine CLIV (September): 620–25.

Sethares, William A. 2017. “Automated Creation of Weave Maps.” In Counting Vermeer: Using Weave Maps to Study Vermeer’s Canvases, edited by Johnson, C. R. Jr. and W. A. Sethares. The Hague: RKD-Nederlands Instituut voor Kunstgeschiedenis. http://countingvermeer.rkdmonographs.nl.

———. 2017 “Computer-Assisted Manual Thread Marking.” In Counting Vermeer: Using Weave Maps to Study Vermeer’s Canvases, edited by Johnson, C. R. Jr. and W. A. Sethares. The Hague: RKD-Nederlands Instituut voor Kunstgeschiedenis. http://countingvermeer.rkdmonographs.nl.

Van Tilborgh, Louis, Tieo Meedendorp, Ella Hendriks, Don H. Johnson, C. Richard Johnson, Jr., and Robert G. Erdmann. 2012. “Weave Matching and Dating of Van Gogh’s Paintings: An Interdisciplinary Approach.” The Burlington Magazine CLIV (February): 112–22.




'Hunting for Weave Matches: Computation in Art Scholarship' has 1 comment

  1. February 21, 2018 @ 12:06 pm Table of Contents: Issue Twelve /

    […] Hunting for Weave Matches: Computation in Art Scholarship C. Richard Johnson, Jr. and William A. Sethares […]

    Reply


Would you like to share your thoughts?

Your email address will not be published.

*
To prove you're a person (not a spam script), type the security word shown in the picture. Click on the picture to hear an audio file of the word.
Anti-spam image

Images are for demo purposes only and are properties of their respective owners. ROMA by ThunderThemes.net

css.php
Need help with the Commons? Visit our
help page
Send us a message
Skip to toolbar