![]() SumoLogic runs using a basic MapReduce philosophy and is sensitive to inefficiency. This way, teams can create alerts and be able to control them without relying on a specific employee. Because SumoLogic has a limit on seats, we create individual accounts for our front-line technicians (SRE, Data Systems, Infrastructure Engineering), and team logins for over 250 software engineers and support staff. Ping will connect to your LDAP service and users can login directly with their LDAP credentials. We use Ping Identity for many services, including SumoLogic. We require our users to identify their volume per hour and consult with SRE before enabling log emission to Sumo. This is a great way to help curtail unnecessary loglines from eating up your licensed data volume cap. Furthermore, SumoLogic provides configurable user roles, which allow things like limiting regular users to only being able to search application logs, while only sysadmins have access to categories like sysinfo (/var/log/messages) and authpriv (/var/log/secure).Īlthough rsyslog can do its own log filtering, each Collector also has the ability to filter logs at the Collector itself, meaning loglines matching the filter never get sent out of the Collector to SumoLogic servers. In addition to user-defined application logs, every host also sends critical Linux OS logs by default, which are classified as different sources. In our case, different syslog facilities are identified in each Collector by the SumoLogic concept of a source. Another great feature of rsyslog is its ability to spool logs - with a configurable size - if the Collector is slow or unavailable. ![]() If an application can send directly to syslog and be configured with a syslog facility, we’ll use that instead of the rsyslog file tailing. For instance, we send both network device and application logs through local4, but /var/log/messages through local2. One advantage to using rsyslog is its ability to do “file tailing.” This allows us to send any iteratively updated file to Sumo, while retaining the ability to send directly through traditional syslog facilities. Early on we learned that lesson the hard way.Įvery host we operate has a rsyslog configuration that uses a specific syslog facility to forward various types of logs to the SumoLogic Collectors. One word of caution: it is important to make sure you assign enough heap memory to the Collector (a Java process) to account for the amount of logs it funnels. Although proxy support is now provided in the Collector configuration and should allow individual nodes to emit to Sumo, we still prefer a single Collector in each of our datacenters for ease of managing thousands of instances. In addition, some of our internal IS services have their own Collectors. ![]() This is How We Doįrom a user perspective, SumoLogic comes in two parts: a Collector local to your hosts and the main UI where searches and configurations are done.Īt OpenX, each Collector serves as a central syslog server, and we employ a single Collector in each of our physical datacenters. Not only does SumoLogic enable our Technical Operations teams to perform rapid analysis and developers the ability to monitor and troubleshoot, but it also gives us critical real-time views of our systems and alerts us when things go wrong. It allows us to give the pleasure of beautiful logs to more than just operations and development. In fact, one of the top users of SumoLogic is our customer support team. Many of our employees do not have direct access to logs. Why do we like log aggregators? For one, we have thousands of nodes to follow, and gaining trending insight from logs is important for analysis and maintaining the health of our complex system. With the growing scale of operation at OpenX, gaining intelligence from them ceased to be a manual task long ago when we adopted SumoLogic as our log aggregator. They provide feedback, allow us to view different angles of history, give baselines to identify aberrations, and highlight anomalies.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |