IT Briefcase Exclusive Interview: Getting Into Google’s Buganizer

Security researcher Alex Birsan was able to gain access to some of Google’s most critical and dangerous vulnerabilities by finding and exploiting a series of flaws in the company’s internal bug tracker called Google Issue Tracker or “Buganizer.” By spoofing a Google corporate email address, he was able to gain access to the back-end of the system and thousands of bug reports, including those marked as “priority zero,” vulnerabilities which are the most severe and dangerous and enable the greatest amount of damage. Craig Young, computer security researcher at Tripwire, discusses what happened, why someone would want to do this, and what information was exposed, including what organizations can do to mitigate risk from such events.

• Q: What is a bug tracker?

A: ‘Bug tracker’ is the name given to software used by organizations to effectively communicate details of software defects across teams. These systems are critical for making sure issues are triaged, fixed, and properly tested. The reports themselves may come from a variety of sources including user support requests or internal testing. It is also important to note that many bug trackers employ access controls such that users can only see information which is relevant to their role in the organization.

• Q: Was Google’s bug tracker hacked?

A. Yes, but not necessarily by a hacker with malicious intentions. Google actively encourages so- called ‘White Hat’ hackers to test their systems and report back weaknesses. In this case security researcher Alex Birsan found and exploited a series of vulnerabilities which ultimately gave him access to bug reports he should not have been able to see. This included access to sensitive security bugs including those with the highest criticality (P0).

Birsan reported these issues and Google promptly deployed fixes within a matter of hours.

• Q: Did Google really pay someone for hacking their system?

A: Google’s vulnerability rewards program (aka bug bounty) paid Birsan more than $15,000 for these reports. Although Google was one of the early adopters of the bug bounty model, it has since become a rather common practice that tech companies will use bug bounty programs to crowdsource security improvements. The goal of bug bounty programs is to encourage users that are motivated to look for and report vulnerabilities back to software vendors rather than selling information to a third party or simply ignoring the issues. This is in stark contrast to the historically rocky relationship many software firms have had (and some still have) with researchers. In my experience with their bug bounty program, Google has always been very generous with their bounties, but in this case, I actually wonder why they didn’t pay a larger bounty for a set of bugs which could seriously undermine the security and privacy of millions of people.

• Q: What information was exposed through the hack?

A: The system Birsan compromised contains issue reports from some but not all Google products. For example, an issue I reported in Android earlier this year was entered into the Buganizer system while security reports I filed for the Chrome web browser went into a Monorail bug tracker specific to Chromium. Reports submitted to the Google Vulnerability Rewards Program (VRP) for Google’s web properties go through yet another workflow and it is not clear to me whether Buganizer may be involved in this. Based on Birsan’s report, however, we know that data about critical vulnerabilities in at least some of Google’s products or services is stored in this system.

• Q: Why would someone want to hack into a bug tracker?

A: Hacking has become a major tool for a wide range of threat actors including corporate spies, government spooks, criminal enterprises, and law enforcement agencies. While most hacks tend to leverage human error, some of the most sophisticated attacks make use of mistakes in software for which there are no available protections. This type of flaw is known in the industry as a 0-day flaw because there are 0 days during which a victim could have properly remediated the issue. Compromising a bug tracker can give attackers the kind of detailed information they need to exploit these 0-day flaws in software used by their adversaries. Even vulnerabilities which have already had fixes released are often quite valuable as a result of the poor patching practices employed by many organizations.

• Q: Is this the first instance of something like this happening?

A: In recent years, we’ve learned of several bug-tracker related issues affecting prominent software vendors. The most notable of such events is when it was revealed that BugZilla, the bug tracker developed by the Mozilla, would truncate email addresses in a way that was suitable for impersonation attacks. This was a particularly tricky situation due to the fact that BugZilla, an open source project, has been widely used throughout the industry making it incredibly difficult to coordinate a patch release that minimized exposure to users. The maintainers knew that hostile users would start exploiting bug trackers the instant the flaw became publicly known. In order to thwart this, BugZilla’s developers gave a heads up about the bug to a limited number of trusted BugZilla admins. Nonetheless, it is quite possible that various organizations who were not lucky enough to be forewarned were exploited before patching.

While it is unclear if any malicious attackers had leveraged the BugZilla vulnerability before it was publicly known, a recent report from Reuters claims that an unknown threat actor successfully compromised Microsoft’s bug database back in 2013. Microsoft, however, has not confirmed this report and no information was offered about how attackers may have accessed the system or how long they may have had access before being detected. Functional exploits for various Microsoft products have been valued in the six-figure range by grey-market exploit brokers like VUPEN and Zerodium, but the financial incentives can be far greater by using them in an offensive attack campaign. (Think industrial espionage, blackmail, or plain old ransomware on a global scale.)

• Q: What can organizations due to mitigate risk from such events?

A: Software firms should employ layered access controls on top of bug trackers containing sensitive information. Details of security bugs should not be visible by everyone and ideally should be kept in a separate bug tracker. The use of a secondary bug tracker for security information minimizes the risk of typical privilege escalation attacks and at the same time makes it much easier to recognize potential attacks on that tracker.

The risks posed to other organizations and individuals, however, is far harder to defend against. The best anyone can really do is follow good security practices like patching, 2-factor authentication, exploit mitigation, education, etc. These types of efforts help put roadblocks ahead of attackers as well as to reduce the impact an attacker can make.

Craig Young is a computer security researcher with Tripwire’s Vulnerability and Exposures Research Team (VERT). He has identified and responsibly disclosed dozens of vulnerabilities in products from Google, Amazon, IBM, NETGEAR, Adobe, HP, Apple, and others. His research has resulted in numerous CVE assignments and repeated recognition in the Google Application Security Hall of Fame. Craig’s presentations on Google authentication weaknesses have led to considerable security improvements for all Google users. Craig won in track 0 and track 1 of the first ever SOHOpelessly Broken contest at DEF CON 22 by demonstrating 10 0-day flaws in SOHO wireless routers. His research into iOS WiFi problems more recently exposed CVE-2015-3728 that could allow devices to inadvertently connect to malicious hot spots. Craig has more recently turned his attention to a different part of the wireless spectrum with research into home automation products as well as RFID/NFC technology.