Businesses are always trying to find the right tools to meet their needs and make monitoring, logging, and failure analysis efficient and fast.

Zebrium You may prioritize Prometheus or ELK, but Zebrium

can also be prioritized.

The new startup was named one of Gartner’s “25 Enterprise Software Startups to Watch in 2020.”

Speaking of best practices, Zebrium also recently helped Sweetwater reduce event tracking time from 3 hours to minutes. Zebrium can even uncover hidden problems that went undetected before. This is a great feature because it can help identify issues before they affect customers.

So what sets Zebrium apart from the competition?

Zebrium uses artificial intelligence to spot issues and automatically discover root causes, while all other tools rely on users manually adding rules. Zebrium can also be used as a standalone log management platform or integrated with ELK Stack or other log managers.

Advantages: easy to install and use; Just copy/paste the custom HELM or kubectl command; Automatically detect problems and root causes without manual rules; Can be used as a standalone log management tool or as a machine learning plug-in for an existing log management tool such as ELK Stack.

Cons: The free plan is limited to 500 MB per day and reserved for 3 days; Kubernetes, Docker, and most common platforms are supported, but Windows is not supported.

Sematext

log management and application performance monitoring solutions. Smatex provides full-stack visibility into system status.

Sematext is not limited to Kubernetes logs, but can also monitor and Kubernetes (based on metrics and logs).

Collected logs are automatically parsed/structured according to several known log formats, and users can also provide custom log patterns. It also exposes the Elasticsearch API, so any tool used with Elasticsearch, such as Filebeat and Logstash with Sematex, can be used as a variant of ELK or with the native Sematext ecosystem.

This tool helps you create specific rules to monitor specific situations and catch anomalies. With Sematex’s comprehensive real-time dashboard, customers can control and monitor all services.

Pros: Integration with other Sematext cloud tools; Configurable overrides to prevent logs from being accepted, thus controlling costs; ELK’s flexibility.

Cons: Sematext widgets and Kibana cannot be mixed on dashboards; Custom parsing needs to be performed during log shipping, and Sematext only parses syslog and JSON on the server side; Also the tracking system is weak, but improvements are being planned.

Loki

Loki

is a multi-tenant and highly available log aggregation tool inspired by Prometheus. This tool helps collect logs, but users need to create manual rules for them. Loki works with Grafana, Prometheus, and Kubernetes. Loki can improve the efficiency of internal processes. For example, it saved Paytm Insider 75% in logging and monitoring costs. Loki doesn’t index your log content, but only builds a set of labels for each event stream, so it’s very efficient.

Pros: Huge ecosystem; rich visualizations; Efficiency is improved due to unindexed log content.

Cons: Not optimized for Kubernetes log management; Extensive manual work using schema rules; Lack of content indexing can limit search performance.

ELK ELK

is

the most well-known open source tool for log management and is an acronym for Elasticsearch, Logstash, and Kibana. Each component handles a unique part of the logging process; Elasticsearch is a powerful and scalable search system, Logstash aggregates and processes logs, and Kibana provides analytics and visual interfaces to help users understand the data. Together, they provide a comprehensive logging solution for Kubernetes. However, there are many other variants of the ELK Stack, such as the EFK Stack, which consists of Elasticsearch, Fluentd, and Kibana.

ELK is used by many large companies such as Adobe, T-Mobile, and Walmart, so it can prove its producibility. So ELK is a reliable and proven tool. But he brings with it the complexity and significant resources required for the job.

Pros: ELK is well known and has a large community; Very wide platform support; Rich analytics and visualization capabilities in Kibana; Logs and manually defined alert rules can be analyzed for sophistication.

Disadvantages: difficulty maintaining scale; Requires a lot of tuning, especially for large environments; Substantial resource requirements; Some features require a paid license.

Fluentd Fluentd

is a

cross-platform, open-source data collector that provides a unified logging layer, but it is not a standalone log manager. A popular tool, it has more than 5,000 customers such as Atlassian, Microsoft, and Amazon. See these big customers prove its reliability and performance. In addition, Fluentd creates a unified logging layer that can help use data more efficiently and iterate quickly in software. It can process 120,000 records per second.

Pros: Large community and plugin ecosystem; Unified log layer; Proven reliability and performance. Can be installed in less than 10 minutes.

Disadvantages: difficult to configure; Limited support for transforming data; Not a complete logging solution.

Conclusion

You might ask why Prometheus is not included in the list because this article focuses on log collection tools, while Prometheus only deals with metrics and does not support logging.

So, if you’re not good at manually searching logs, or aren’t comfortable building and managing alert rules, try Zebrium, which is based on machine learning algorithms. This can save a lot of time and get rid of the tedious task of creating a large number of rules.

If you’re looking for something more mainstream and know which rules to create, try Loki or Sematext, which are very efficient tools.

If you want to use

log monitoring in the public cloud, you may want to use a service provided by your cloud provider, such as AWS’s CloudWatch, which only powers businesses in their own cloud.

If your logs have multiple or specific sources, try using Fluentd and its unified logging layer, but you still need a logging tool.

Original: https://medium.com/codex/5-top-kubernetes-log-monitoring-tools-d8c0494deb30