[ODFPlugtest] plugfest test file results

[robert_weir at us.ibm.com]

plugtest-bounces at opendocsociety.org wrote on 11/05/2010 04:20:34 AM:

> 
> The hard part is to verify that a document renders as it should 
> render. Having 
> a button in OfficeShots that allows people to say: "this render is 
> what I would 
> expect" would be nice, along with a way of discussing on the particular 
> render.
> 


Right.  I'm assuming that the person creating the test case is not an ODF 
or XML expert.  But they do know their editor.  So they are creating a 
document that "looks right" to them.  This does not guarantee that the 
document is valid or conforms to ODF.  But it does mean that at least on 
their machine, it looks right. 

So one thing we could do is collect the name of their editor, the version 
and the OS and associate that with the test document.  Ditto for the 
verification step. 

Aim for self-describing test documents, e.g., a document that has text 
saying, "This document shows 12-point bold text, center aligned".

If another rendering shows this document differently, we don't know why. 
It could be because:

1) The original document was invalid
2) The original document took advantage of non-standard extensions to 
accomplish its goal
3) The feature tested is implementation-defined in ODF
4) The ODF specification is ambiguous on what is required
5) The rendering application has a bug

>From the interop view, these are all interesting possibilities.  But it 
would require further analysis to determine the "root cause" and that 
probably cannot be automated.  But even the rough list of "unexplained" 
errors is valuable to have.

-Rob