> A syslog server is a foundational tool for centralized log management in modern IT environments
I'd very much recommend a more modern log stack than a traditional syslog server.
There are many articles covering the limitations of Syslog. Better to replace that component by utilities like OpenTelemetry and JSON structured logging.
I disagree. I work in a large environment, and rsyslog is where 90% of data goes to first. It can keep up with millions of messages per second, route them to higher order services for indexing (bigquery, splunk, elastic etc etc). Has rules engines, encryption, supports multiple protocols and obviously has TLS too. You can surely augment with otel and such where you can, but syslog is uhhhh, deployed in so many places that it would make an average app developer's head spin when all they're used to is application logging in a controlled structured place in their silo.
On the other hand, traditional syslog is UDP based, so as soon as the receiver experiences CPU or I/O starvation and its receive buffer overflows, it will begin dropping messages. That's not great for observability, and may well be impermissible at many sites that need end-to-end log integrity (e.g. audit logs).
cheaply dropping log msgs you cannot handle is absolutely essential for an observability system - otherwise excess load can take down the logging infra which can (if msgs aren't dropped) take down the prod network/app trying to send reliable log msgs.
And the number of k8s envs that log stdout through them into.... more rsyslog, it's truly everywhere. Plus all the sidecar containers deployed that shuffle app logs, lots of syslog there, its so lightweight and simple and reliable. I watch all the gyrations people go through to achieve the same result, and it's always changing, hurts my brain thinking how much time they waste
I've been down this road many times over the years, and each new promised land of logging always fails to replace syslog for me. Nothing else is as widely-compatible and performant. Fortunately, there are some great tools out there to modernize the network-admin experience: syslog-ng is a favorite of mine.
Disagree. JSON logging is fine, but make it line based and just log to a syslog server. Then I can route it wherever I want (or to multiple places) including to a simple file on a disk that I can actually inspect, rather than having to use an API.
The problem with modern stuff is it doesn't do the very basics. Sometimes I really do want UDP dumping out into a file on another part of the network. The modern setups forget how to do this.
I saw one place that had the logs going into a database. On the same connection as the app-data. So, when the transaction failed, the logs also didn't get written. LMAO. I made them do syslog in their code, which for some of the devs was a mind-blowing. They were amazed at that we could just barf text quick&lightweight over UDP.
Mongo stopped having major data loss issues after a decade - my understanding is it's pretty stable now. You might hate the company for beta testing on your data but that's a separate issue.
I've never been in the sales room. I've heard from multiple corps that Splunk is quite expensive. I've been to one place that ran its own Elastic stack, which was a bit of cost in its own right because they had to run all the infrastructure and storage themselves, and had two FTEs whose main job was keeping the thing running. I don't know what Splunk offers from a UI/features perspective that Kibana couldn't do.
There's far cheaper alternatives to Splunk that are still a step up from traditional syslog.
KubeCon over the last couple of years was showing the market was a glut with Observability vendors which is just time series and log management. (traces are logs with a span id).
> A syslog server is a foundational tool for centralized log management in modern IT environments
I'd very much recommend a more modern log stack than a traditional syslog server.
There are many articles covering the limitations of Syslog. Better to replace that component by utilities like OpenTelemetry and JSON structured logging.
You can run a single binary version of Loki https://grafana.com/docs/loki/latest/get-started/deployment-...
Or use something like the Otel collector to send your logs to a remote host.
I have done my fair share of rsyslog and syslog-ng.
I would not say a "Syslog" server belongs in a modern stack.
I disagree. I work in a large environment, and rsyslog is where 90% of data goes to first. It can keep up with millions of messages per second, route them to higher order services for indexing (bigquery, splunk, elastic etc etc). Has rules engines, encryption, supports multiple protocols and obviously has TLS too. You can surely augment with otel and such where you can, but syslog is uhhhh, deployed in so many places that it would make an average app developer's head spin when all they're used to is application logging in a controlled structured place in their silo.
On the other hand, traditional syslog is UDP based, so as soon as the receiver experiences CPU or I/O starvation and its receive buffer overflows, it will begin dropping messages. That's not great for observability, and may well be impermissible at many sites that need end-to-end log integrity (e.g. audit logs).
cheaply dropping log msgs you cannot handle is absolutely essential for an observability system - otherwise excess load can take down the logging infra which can (if msgs aren't dropped) take down the prod network/app trying to send reliable log msgs.
Audit logs are a distinct feature.
Even SC4S, splunk's docker appliance for turnkey syslog uses rsyslogd.
Edit: being pedantic -- it's syslog-ng actually.
And the number of k8s envs that log stdout through them into.... more rsyslog, it's truly everywhere. Plus all the sidecar containers deployed that shuffle app logs, lots of syslog there, its so lightweight and simple and reliable. I watch all the gyrations people go through to achieve the same result, and it's always changing, hurts my brain thinking how much time they waste
I've been down this road many times over the years, and each new promised land of logging always fails to replace syslog for me. Nothing else is as widely-compatible and performant. Fortunately, there are some great tools out there to modernize the network-admin experience: syslog-ng is a favorite of mine.
Disagree. JSON logging is fine, but make it line based and just log to a syslog server. Then I can route it wherever I want (or to multiple places) including to a simple file on a disk that I can actually inspect, rather than having to use an API.
The problem with modern stuff is it doesn't do the very basics. Sometimes I really do want UDP dumping out into a file on another part of the network. The modern setups forget how to do this.
I saw one place that had the logs going into a database. On the same connection as the app-data. So, when the transaction failed, the logs also didn't get written. LMAO. I made them do syslog in their code, which for some of the devs was a mind-blowing. They were amazed at that we could just barf text quick&lightweight over UDP.
Yeah, for a modern large scale distributed system both the client api and implementations are pretty bad.
An intro article on syslog servers in the year of our Lord 2026?
Did you know you can replace your noSQL db with VSAM too (honestly better than mongo).
I'd reckon that even in this modern age there are on the order of 10^9 people who don't know what a syslog server is.
Mongo stopped having major data loss issues after a decade - my understanding is it's pretty stable now. You might hate the company for beta testing on your data but that's a separate issue.
We moved to splunk now and mostly happy with it, mix of windows/linux/etc logs.
But with AI, I can see opensource alternatives getting better.
I've never been in the sales room. I've heard from multiple corps that Splunk is quite expensive. I've been to one place that ran its own Elastic stack, which was a bit of cost in its own right because they had to run all the infrastructure and storage themselves, and had two FTEs whose main job was keeping the thing running. I don't know what Splunk offers from a UI/features perspective that Kibana couldn't do.
I agree (as the owner of an elastic stack at work). A second FTE sounds like a luxury though..
"Splunk that runs locally with a worse UX, but on balance doesn't eat all the resources you throw at it and doesn't charge you a kidney."
There's far cheaper alternatives to Splunk that are still a step up from traditional syslog.
KubeCon over the last couple of years was showing the market was a glut with Observability vendors which is just time series and log management. (traces are logs with a span id).