We recently completed a research work that ended with a bunch of negative results. Even if negative, we thought the results we obtained were valuable because, in our opinion, were not obvious (in fact we wanted to “prove” that the variables we studied were indeed a positive factor).
The next question was, so, where do we try to publish this?. In theory, anywhere. I mean I’ve never seen a conference or journal in Software Engineering explicitly forbidding you from submitting negative results. My problem is that I’ve not seen any that explicitly encourages you to submit them either (I’d love to be wrong so please please correct me if you can!).
There are quite a few journals in other areas specifically devoted to publishing negative results (e.g in BioMedicine, Ecology, Physics,…) but nothing in Computer Science except for this “Forum for Negative Results” published as a section of the Journal of Universal Computer Science. Nothing for Software Engineering.
That´s why I´m calling here the need for a Journal of Negative Results in Software Engineering. Anybody else thinks this would be a good idea?
(I´d also settle down for a more explicit encouragement, and acceptance, of negative results in existing conferences / journals; I do believe that many people are afraid of submitting their negative results and we are losing all those findings; as a good first step we ended up organizing this special issue with six negative results in Software Engineering published with EMSE )
I think this is a good idea. Specially since negative results are more likely to be statistically significant and having a venue which publishes them would be encouraging. Another aspect is reproducibility studies, that is also an area where software engineering community can do better.
Completely agree on the need for more coverage for reproducibility studies as well. They have even less chances to be published, I´m afraid. A nice paper on this would be: Jonathan Lung, Jorge Aranda, Steve M. Easterbrook, Gregory V. Wilson: On the difficulty of replicating human subjects studies in software engineering. ICSE 2008: 191-200