Android Lint is a static analysis tool for Android which looks for 225 (261 as of May 2016) different types of Android bugs. The potential bugs span a number of categories:
The public usage documentation for Lint is here: http://developer.android.com/tools/debugging/improving-w-lint.html Current StateLint is integrated in a number of tools:
Non-Goals
Goals
Whoops, That One Slipped ThroughA classic problem for error checking tools is that they’re available, but that users don’t use them. Yes, they may spot a squiggly warning line in the editor if they happen to be looking at code in the same neighborhood. But there are some checks which aren’t highlighted on the fly and require the user to explicitly check for warnings (in IntelliJ, that action is Analyze > Inspect Code..). Furthermore, even if the user sees the warning, and intend to fix it, they may get distracted and forget. Lint has a solution for this. While of course it’s integrated with Gradle in the same way that other static analysis tools are (findbugs, checkstyle, etc), and will generate reports if you run the “check” target, it goes one vital step further: When you generate a release build, whether you ask for it or not, lint will be run too, in a special mode where it only looks for issues with “fatal” severity. This is usually a much smaller set of checks than a normal full lint run (currently 36 our 225 lint checks have severity fatal), so it’s faster than a normal lint run (and given that release builds often perform slow Proguard shrinking and obfuscation too, the additional overhead of fatal lint checking is minimal when compared to the overall build time.) If any fatal issues are found during a release build, the release build is aborted! Yes, users can turn off this feature, but it’s on by default, and helps Android developers avoid shipping critical problems just because they forgot to run Inspect Code one more time after that last simple innocuous edit… ExamplesHere’s a (non-exhaustive) list of some of the checks lint performs:
ArchitectureThe above goals have a number of implications for the architecture of lint:
Performance, Performance, PerformanceThe top challenge for lint is to be highly performant, especially when running in the IDE. This is addressed through a number of approaches:
Annotation-Based ChecksThe support library now ships with a number of annotations which express additional metadata about code. These are all documented here: https://developer.android.com/studio/write/annotations.html. These annotations let you for example indicate that
These annotations are also available in the framework (in the android.annotation namespace instead, but hidden). The SDK extracts these annotations, converts them to the corresponding support annotation, and ships these in an extracted form both with the platform-tools (for command line lint usage) and with Android Studio (for IDE usage). Lint has special support for checking for these annotations. For every single method call, it finds the corresponding method and checks whether it, or any of its super classes, are annotated with a support annotation (either directly on that code, or in the external annotations database, as discussed above). This is also done for field references, and class references. For each support annotation constraint it finds, it then perform analysis to discover whether the annotation constraint is met. Typedef Annotations (@IntDef) and External AnnotationsThe @IntDef annotations check is one of the annotation based checks, but it has some interesting special considerations. First, @IntDef allows developers (and the framework) to use a primitive integer instead of an enum to represent that a parameter must be one of a specific set of values, offering compile-time safety to the same extent that an enum does -- but without the same overhead. (Much has been made about the number of extra bytes required by an enum in the .dex file, but another consideration of enums is that the runtime class also needs to initialize all constants as objects.) The main benefit of an @IntDef is that you can have two unrelated constants that have the same value, and while the compiler will happily let you pass in one instead of the other, the typdef annotation knows the specific allowed constants and will flag any other constants, whether the values are the same or not. However, annotations in .class files cannot record “references to constants” in this way; they can only contain a limit set of specific types of data: primitives, Strings, Class instances and enums. Therefore, if we look at the .class file for a typedef enum, it only contains the actual values of the constants, not the references to the fields that contained the constants -- and with only the values, lint can’t actually do its job down the line. Therefore, the typedef annotation needs to either be local to the project (such that lint can look at the AST node for the typedef annotation itself), or we need to record the data in a separate mechanism: an external file. That’s what lint does. When Gradle is compiling libraries (such as the support library), it runs a special “extract annotations” task, which looks for typedef annotations, and when it finds any, it records these annotations in a special file inside the AAR file. Lint, when analyzing projects, will look at all the AAR files of the project’s dependencies, see if any contain the special external annotations file, and if so, include it in its annotations lookup described above. Flow AnalysisControl Flow AnalysisLint has multiple implementations of control flow analysis: one operating at the bytecode level, for the .class file based checks, and the other for Java AST based analysis. One obvious use for control flow analysis in lint is the WakelockDetector, which looks at wakelock acquire and release calls, and makes sure there is no escape path via exceptions that can lead to the wakelock release being skipped. However, the most “important” control flow check in lint is related to the API check, because it avoids a lot of false positives! Lint’s API check looks at every method call and field reference, and makes sure that the call was not introduced in an API level higher than the minSdkVersion supported by the phone. However, what if that code is surrounded by an explicit version check? In the below, we don’t want myLollipopCall() to be flagged as invalid, since clearly it can only be executed if we’re on Lollipop: if (Build.VERSION.SDK_INT > LOLLIPOP) { myLollipopCall(); } Checking this is not as simple as just looking to see if a method call is surrounded by a simple if check like this (though usually that’s where the lint check is). It’s not even enough to look at all outer if statements. We also need to handle scenarios like this: if (Build.VERSION.SDK_INT <= LOLLIPOP) { return; } myLollipopCall(); ...and so on. Hence, lint will build up a control flow graph to analyze the control flow in these cases such that it can properly be silent about API “violations” that are found to be unreachable on older platforms. 2017: There's a new global call graph for use by lint checks such as the inter-procedural thread checker. This is still experimental. Data Flow AnalysisLint performs data flow analysis for a number of checks. Just as for control flow, this is implemented in two separate ways: one for bytecode analysis, and one for AST based analysis. Examples of AST-based data flow analysis:
Examples of bytecode based data flow analysis:
Constant FoldingLint performs “constant folding” on the AST, including for symbols referenced from other compilation units. For example, if in one class you define public static final String LOG_TAG_PREFIX = “MyReallyLongLogTag”; and then from some other class you do this public static final String LOG_TAG = OtherClass.LOG_TAG_PREFIX + “_Parser”; Lint will flag a logging call using this tag as invalid, because the LOG_TAG is too long (the max allowed is 23 characters), and it computed this on the fly in the editor by performing constant evaluation: Implementation DetailsLintClientLint is highly embeddable; it’s been integrated natively in Android Studio/IntelliJ, Eclipse, the Android Gradle plugin, Jenkins, as a command line tool (“lint”) distributed with the SDK -- and this tool is itself used to integrate lint in tools like Tricorder and Maven. LintClient is the abstraction which makes this possible. When lint needs to read a file, it doesn’t do that directly, it asks the LintClient to do it. When lint is running in an IDE for example, this lets the IDE return the current, possibly modified and unsaved changes of the given file, not whatever happens to be on disk, which is vital when lint is checking code while the user is typing. When lint finds errors, it also reports these back to the LintClient. This lets for example an IDE highlight text ranges in a source file, whereas for something like the Gradle plugin, it can instead aggregate and sort the errors for a command line report, an HTML report, or an XML file to be analyzed by something like a Jenkins plugin. Issues versus DetectorsA particular class of bug is called an Issue. An issue provides information about this class of bug, such as
Note that an issue is just data; it’s not the class responsible for finding that problem. An issue is found by a Detector. The advantage of separating out the process of looking for an issue from the issue itself is that a detector can often look for related issues at the same time, using the same underlying scan and data structures. As an example, the detector which analyzes formatting strings and reports mismatched parameter types with string formatting argument types, also finds formatting strings that should probably be using plurals instead. These two issues are completely unrelated from a bug perspective, and have different categories and severities, but they’re both identified by a single detector which has identified a given string in a resource file as used for formatting. The issues are all aggregated in an IssueRegistry. Out of the box, there is one special issue registry: BuiltinIssueRegistry, which provides all the built-in lint issues, currently 225. However, lint also supports “custom lint rules”: additional lint rules that are not part of the base distribution. There are two primary use-cases for this:
Lint will also look in some known locations (such as ~/.android/lint and $ANDROID_LINT_JARS) for additional custom rules. This can be used for company or personal lint rules across any project that is loaded. To provide a custom rule you basically place your Detector class as well as an IssueRegistry which points to the issues into the .jar, and point to it with a manifest attribute. At runtime, lint will find all the issue registries, it will use JarIssueRegistry to load issues from a jar file, and it will merge them all together with a CompositeIssueRegistry. Rich TextLint’s messages (not just issue explanations, but the error messages themselves, etc) are all in a special “markdown-like” format, where you can surround characters with asterisks (*) to make words bold, with backticks (`) to format the text as a symbol, etc. Internally, this is referred to as TextFormat.RAW, but it can also be formatted to TextFormat.TEXT or TextFormat.HTML. It’s typically formatted to plain text in command line output, but in the IDE, it’s formatted to HTML, which allows the error message to look like this: Here the hyperlink is highlighted as a link, and (though it’s hard to spot) the “contentDescription” attribute mentioned in the error message is highlighted with a code font instead of a normal text font. Binding Issues and DetectorsEach issue is mapped to a specific detector. When lint rules, it checks which issues are enabled (which can be controlled for example by IDE metadata such as inspection settings, as well as project metadata, such as a Gradle lintOptions section, as well as by a config file named “lint.xml”). From this it finds out which detectors need to be run. If none of the issues associated with a detector are enabled, that detector doesn’t have to run at all. There is actually one extra level of indirection in the mapping from issues to detectors: an Implementation. When lint initializes the detectors to run, it asks an issue’s implementation for the associated detector. This indirection allows a particular tool integrating lint to substitute out the detector used to find a specific issue. For example, the API Check needs to resolve every single call and field reference, and it does this via bytecode analysis. However, when running inside IntelliJ, we swap out the implementation for a custom detector which can take advantage of certain persistent data structures to do the job much more efficiently. Scopes & Analysis ScopesA scope indicates what files are required for a given detector to do its job. For example, for a lint check which looks for performance problems in onDraw and onMeasure calls, the scope would simply be Scope.JAVA_FILE. There are two purposes for scopes:
For example, consider the Eclipse integration. When the user edits a file, and then saves it, Eclipse will compile the file immediately and create a corresponding .class file. When lint is integrated in Eclipse, like the other tools it operates on the save action and runs lint. It can now provide both the .java and the .class versions of this file, so it passes in the set of scopes [JAVA, CLASS]. Now lint knows that it can run all the lint checks that require either JAVA, or CLASS, or both. However, just because a lint check applies to for example both XML resource files and Java files, doesn’t mean that it needs to look at both in order to report errors. In some cases, it can independently check both. The PrivateResourcesDetector for example can independently look at R.type.name references in Java files and @type/name resource references in XML files and indepedently warn if it finds that a given resource usage is problematic. To solve this, the Implementation (which maps issues to detectors and specifies the scope required for the detector to handle a given issue) can also specify “analysis scopes”; this is a list of scopes that the detector can independently handle. The LintClient has several lookup methods which are useful for incremental operation. For example, it has a facility to look up resource references, which in the IDE is already a persistent data structure used for example for layout rendering, and with that facility, lint’s “wrong cast” detector, which makes sure that after a findViewById lookup the cast to a class is compatible with the types actually found for the id in the layout, can operate incrementally when the user is editing the Java class. As soon as lint sees the cast, it consults the resource repository, finds the associated places the ID is referenced in layouts and makes sure they’re compatible with the cast. Currently, 188 out of lint’s 255 lint checks can run incrementally, when taking analysis scopes into account. ProjectsLint uses the term “project” for what we now call a “module” in Android Studio; its usage is similar to the terminology in Eclipse and in gradle itself. A project can depend on other projects. When you depend on a library, that library is also considered a project by lint -- and lint will look at it (it has to, to for example correctly compute unused resources accurately). However, lint keeps track of whether it’s your code or upstream code you’re just depending on, and it won’t flag warnings in code that you’re not responsible for. There are different types of projects: Android app projects, Android library projects, and non-Android projects (e.g. plain Java code; Guava would be an example of this type of library.) Many lint detectors make a distinction between these; for example, there’s a check to make sure you specify whether you want to allow automatic backups or not, but this is only required for an app project, not a library project. A project can also provide a “subset”: a specific set of files (typically 1) to analyze. This is used when lint is run incrementally in the IDE; it still needs the full project context surrounding the file being edited (such that it can consult the minSdkVersion and targetSdkVersion in the manifest, dependencies and so on). Finding IssuesA LintClient can perform lint by creating a LintDriver, and providing it with a LintRequest. The LintRequest basically identifies the root projects to analyze, the available scopes, and the lint client to use. It then invokes the lint driver, which does all the heavy lifting in lint:
The individual detectors will report issues back to the driver, which performs extra checking, such as filtering out reported issues if they are suppressed via annotations, XML metadata, or comments in the source code, as well as ignoring reports for issues that are suppressed (which could be caused by a detector which reports multiple issues and is enabled because at least one issue is enabled, but reported a problem for a disabled issue). Embedding LintAs described earlier, the most important aspect of integrating lint is providing a custom implementation of LintClient. It provides a number of services used by lint:
The LintCliClient is a useful subclass of LintClient for command-line oriented integrations of lint; it handles aggregating and sorting errors, CLI progress reporting, flag parsing, etc. Quick FixesQuick-fixes are automated refactoring operations, typically in the IDE, which can automatically fix the issue identified by lint. These come in two variants:
Note that quick fix operations are not implemented in Lint itself; these are typically handled on the IDE integration side, since as refactoring they typically integrate with the IDE’s native quickfix facility, utilize core IDE services like the ability to lock & modify documents, creating “Undoable” actions, etc. There are currently quickfixes for 52 of lint’s checks in Android Studio. Testing InfrastructureLint has comprehensive unit tests, and a good unit testing infrastructure. It also distributes its testing infrastructure as a separate Maven artifact your custom rules project can depend on for its tests. Each unit tests builds up a test project by describing source files, and then asserting that the output should the report lint would emit when run on the command line. In many of the older tests, the source files passed to lint were stored in separate files and then referenced from the test, but in newer tests I’m including the files inline instead. If you look on the command line this may not look great, but when opened in the IDE, IntelliJ can handle nested syntax highlighting, so it looks like this: In this example, the XML source file, passed to the test is constructed with the xml(source, contents) call; there are others to construct Java source files, etc. For manifest files there are special facilities such that instead of inlining the whole manifest, you can just ask for a manifest with minSdkVersion=14 for example, or with a given set of permissions, and so on. Creating source inputs this way also makes it easy to create synthetic tests. For example, I recently added a lint check to validate App Restrictions, and one of the things the lint check enforces is that the maximum number of nested children is 1000 (this matches the Play Store validator). I don’t have to create a large file with 1000 restrictions and check it into the source tree; I added a loop which added 1001 repetitions in the middle of the file, and then I could check that the lint output complained about the 1001’th element. Lint 2.0We’ve started making plans for lint 2.0 to clean up the APIs and address a number of issues (improving issue registration etc). |
Lint >