Metrics Page Support and Improving Multi-Host Filtering

Does the Metrics page support multiple sources? I'm trying to figure out if sending Metricbeat data from multiple hosts would make any sense but there's no way to filter metrics based on a field (ex hostname).

Currently there is no way to do that on the metrics page. This is bare bones and first version for now. It will get more features slowly and should allow such functionality, For now, you can chart the metrics on the dashboard page with filters. That should work well.

I was going to use charts, but the dashboards are unreadable when you have several hosts. The pie chart tries to fit every single value into it even if it has 0.0n%

you can filter it by host in charts

Yeah, but if you have let's say +10 hosts. It splits the chart in weird ways. Unlike Kibana which lets you choose Top N results. I think it defaults to Top 10

Another option would be adding a new chart type that only filters data, like the "Data Control" visualization from Kibana. That way all the charts/visualizations get filtered by the value on the Data Control

See: https://www.elastic.co/guide/en/kibana/current/add-controls.html#filter-the-data-with-options-list-controls

both suggestions make sense.

As in with a custom query?

Claude do you have any suggestion?

Here are some suggestions to improve filtering and managing metrics from multiple hosts in Metrics:

1. Add filtering controls like Options List, Range Slider and Time Picker that Kibana has. This will allow filtering metrics by hostname, time range, etc. These controls can filter all visualizations on the dashboard.

2. Add a "top N" option to limit the number of series shown in charts. This can help reduce clutter from too many hosts.

3. Add the ability to save filtered dashboards. For example, a dashboard filtered to only show metrics for host1 and host2. This can make dashboards more manageable.

4. Support filters in queries to limit the number of hosts returned. For example, host: host1 OR host: host2 would only return metrics from host1 and host2.

5. Add a "data source" dashboard which shows a list of hosts reporting metrics. This can act as a filtering control where selecting some hosts will filter the main dashboard.

6. Consider adding multiple "dashboard tabs" for the same data, but with different filters applied. For example, have one tab for hosts 1-5, one for hosts 6-10, etc. This may provide more focused dashboards.

7. Add an option to "group by" host (or other field) to show aggregate metrics and then the ability to drill down into specific hosts. This can provide high-level overviews that still allow drilling into detail.

Overall, more robust filtering options, limiting capabilities and better ways to organize metrics across many reporting hosts will help provide a good experience with large data sets in Metrics. Please let me know if you have any other questions!

Gaby I was just trying claude AI to see if it was any helpful. 🤣 Guess it was not.

😂😂😂!

You could still turn to custom SQL mode and and limit 10 at the end to reduce the output.

Hmmm, ok. May be useful to have an option for that in the X, Y, Filter section 😂

it does translate though. You can take the existing chart and open the custom SQL tab and it shows you the entire SQL.

Ahhhh, that's why 😂

a minor annoyance I hope. We need to do this as we have no idea what new fields can come up in custom SQL.

Yeah, makes total sense

Thanks for the feedback. Is it for work or personal stuff?

Already replaced logstash with Vector. In the process of replacing Kibana/ES with ZO

We appreciate your constructive honest feedbacks.

ZincLabs should advertise their releases in the #selfhosted reddit https://www.reddit.com/r/selfhosted

Yes. We plan to do that and a lot more. Just waiting on a couple of things. r/devops is also a great sub.

Awesome! 🙌

QQ -

ELK uses a ton of resources and relies on legacy tech. ZO on the other hand uses modern tech and techniques like parquet files, zstd compression, Vue, etc

Wow, Great points. Thanks a lot.

So when I started playing with ZO, I had to constantly stop/start Vector to change the auth token. Since it changes on each brand-new deployment of ZO. Not a big deal to be honest

Ah, got it.

😂 I think the feature i'm most looking forward is Dark Mode, and enhanced dashboards/metrics. But mostly dark mode 😬

Awesome. BTW, We are getting syslog ingestion directly in ZincObserve without need for vector. API is ready. We just need to build the UI for configuring it.

Looking forward to that! I was going to suggest making the endpoint /syslog instead of `/syslog-server``

It will be a standard log stream for query. You will also have the option to name the stream yourself.

Ahhh, sounds good then :-D

Not yet. We have an open issue on that. It’s a big effort and big priority item. We will get to it soon.

Thanks! 😁