I recently hit an interesting bug in the TestNG plugin for IntelliJ IDEA. The bug was triggered by a null being set on a Properties instance, and after a hour or so working out what was going on I managed to reproduce the problem with the following test setup:
public class SomeTest {
public void someMethod() {
class SomeClass {
@Test(groups = "test")
public void thisTestBreaksThings() {
assert true;
}
}
}
@Test(dependsOnGroups = "test")
public void thisTestTriggersABreak() {
assert true;
}
}
When running the test method "thisTestTriggersABreak", the IDEA plugin searches for any dependent tests which may be needed to bring into the generated suite. In this instance, the plugin finds the @Test method "thisTestBreaksThings" and adds "SomeClass" to the suite, however a PsiClass instance will return null for anonymous and local classes, which then triggers the NullPointerException later on.
The initial patch for the problem was to simply abort the test run when such an invalid test setup is encountered, with the next step is to add an inspection to highlight, and help prevent the problem - however I wondering if this check should be at the annotation/language level rather than at the IDE plugin level.
Currently an annotation's target member can be defined with @Target (see java.lang.annotation.ElementType), but this doesn't allow us to specify any further context - such as "only methods of public classes" (I'm not even sure what syntax would be appropriate, or possible for this). For this functionality I suspect I'd have go to the annotation processor route, but AFAIK this would only work at compile time, so I'd still be needing IDE specific intentions anyway.