We conducted an initial survey of needs about 3D modeling and peer review for archaeological publications. Below you'll see a summary of the results of the survey, including sample comments. The results of this survey served as starting points for parts of the discussion in the first webinar.
If yes, what background and training were important to your ability to carry out these reviews?
- I don't think that I'm necessarily competent to judge the technical quality of a 3D model, especially one that attempts to provide accurate information about existing remains or objects. I have no personal experience with laser scanning, Maya, or the creation of 3D models in professional platforms (though I have used Second Life and photograph-based approaches). So I will not be able to comment on, say, problems with the filling of gaps in the mesh, or the number of polygons a model contains. On the other hand, I use 3D objects and approaches routinely in my own work, and can generally evaluate the usefulness of those produced by others in the archaeological research I conduct. It's a black-box problem: I am eminently competent to comment on the usefulness of the final product and the problems it may have in terms of evidence or authenticity, I am probably not competent to comment on the quality of the technical process that led to the final product.
- Not currently a reviewer, but I think experience working with 3D models in numerous situations would be very important, knowing the limitations and advantages of different methods
- I would need to know more about how 3d modelling of archaeological features is carried out (in the most layperson's terms): the software possibilities (which programs can be used), how much time it takes to produce an image, etc.
- While I have quite a bit of 3D modeling experience, both hands on and supervisory, it is old experience as measured by fast-paced technology time. I don't feel I currently have sufficient grasp of where key assumptions are made in the modeling process or where steps in data transformation take place
- I would want to understand the basics of how a model is put together and if there are any established standards for display (e.g., always include a north arrow and scale, rocks should be shaded a particular way, and so on).
The key points raised were as follows:
1. Legibility - Accuracy - Ease of use / interface
- I am not sure that aesthetic qualities in and of themselves should be review factors, even though this is often how we judge models; I think instead the issue should be how easy it is to understand the model as an illustration of an argument or discussion)
2. Ability to relate models to each other / wider archive
- Connectivity of the model with the primary data on which it is based: can you click through to find excavation records for a layer or detailed photos for a find? Can you get more information about particular features? Can you get easily to the relevant model from references to features in the text of a publication or in an associated database?
3. Quality of source data including media, metadata, contextual data
- Process(es) by which the model was created
- For reconstructions, assumptions and their relation to the evidence presented together with the model
- I'm not entirely sure what kind of metadata standards exist for 3D imaging, but if they do exist then this kind of information should be included and sufficiently robust to allow for a scholar to check the process for potential issues as well as to understand the nature of the image.
4. Contribution of the model to the argument expressed in the publication
- If information derived from a model is cited as evidence for an argument (e.g. "we saw that one of these layers overlay the other only in the model"), whether the model as presented shows, or can be queried to show, the specific information cited as evidence.
- I think the biggest thing is to determine the goal of the 3D image in the context of the article. If the goal is to provide a photorealistic image of some subtle detail and the image is grainy or blurred then this an obvious issue. If on the other hand the image served to advance an argument where clear depiction of relationships is required a more schematic illustration might even be superior to photorealistic images. It may go without saying that some kind of methodological and procedural discussion should accompany the illustrations as well.
- Does it effectively illustrate what the author is attempting to demonstrate or argue? Does the model add to the content or is it simply a novelty?
5. What, if any, meaningful research could be conducted by either the developer or outside researchers using the model?
- Not if the content discusses/advances knowledge pertaining to a research question that is currently relevant
- I really value to ability to look back at archival work. In order to justify the time and money to be put into the production of digital (in this case 3d specialized) content, it seems essential for someone (or some agency) to take responsibility for generating new/compatible version of it for long-term reference.
- I think insisting on the ongoing availability of the model *in the publication* creates an almost insuperable barrier, and will quash innovation. On the other hand, I think a long-term plan for the viability of the *archival information* that produced the 3D visualization -- point clouds, photos, exports to standard file formats with reasonable preservation standards -- should be a requirement for the inclusion of 3D modeling content in publications.
- The raw data (and metadata) are what are important...not the model. Models will change through time as assumptions, archaeological knowledge, and research questions evolve. What is scientifically important is that raw data are preserved so that new and competing models can be created through time (and even at the same point in time). I don't think that these models should be forbidden to be included in publications, simply because the long-term viability is not yet secure. But I also don't know what the work-around for this would be.
- Many comments that this isn’t really a yes or no question. Will be rephrased...
- I think that the journal should be responsible for developing the environment and standardizing it across their own publication, if not across multiple publications.
- In my opinion, author development, within perhaps a broad framework, would allow more innovation/creative use of 3d models than a standardized publication format.
As a reviewer, to what extent is the implementation of metadata/paradata following an accepted standard or schema a requirement for publication of 3D modeling content?
Comments
- For any long-term viability of the model using an accepted standard for metadata is essential
- I'm not sure that this metadata belongs in the publication, however, rather than an archive. At any rate, it should be somewhere. But the issue of approved schemata or standards is tricky
- I think the viability/usability/believability of a model depends on the knowledge of what kinds of parameters went into creating it. This really is what the model is all about. I don't really care how the model turns out, but I really want to know the decisions the author used to create the model. This is what separates a scientific modeling effort from a gaming exercise.
- I would view a well-designed interface in the same way I would view a well-written argument: i.e. required.
- It depends on who has created the user interface. If it is created by the journal then this seems to be blaming the authors for the limitations of the publishers!! If the authors are touting their user interface and it doesn't work or is problematic, then the article would probably require revision. It all sort of depends on what the article is about and how crucial the illustrations are.
- This isn't the job of the reviewer and is an excessive burden on, especially volunteer, reviewers
- Generally, I'd suggest fewer and more substantial revisions. Frequent review of lots of small changes makes is harder to maintain perspective of the wider 'product' and distance from it (as a reviewer)
- I think we're at a watershed with respect to the traditional publishing industry. My considerations of publication venue will still be driven in large part by the academic credit I will be given for it -- which still means the big traditional journals and print books (I also have a lingering feeling that these will have more longevity, despite my digital interests). My considerations of excavation documentation are driven by the desire to document as much as possible, as richly as possible, in the hopes that someone in the future may be able to reuse my information (or even improve it). As a result, I'll collect the data even if I have no idea whether it can be published, especially as the tools become easier and more powerful…
- Personally I am yet to see anything useful to be gained by publishing a 3D model that cannot be more clearly summarised by a decent illustration or photograph.