[Firehol-support] netdata

Wed Mar 16 23:00:21 GMT 2016

I checked what netdata does.

In /proc/stat the 'cpu' line has the total cpu load of the machine.
Just below it, there is load per core.

To calculate the % load using these values, you just calculate the
incremental % of each category versus the total, between any 2
snapshots.

The values are in this sequence:

user
nice
system
idle
iowait
irq
softirq
steal
guest

So, you take 2 snapshots and you find for each of the above the
difference between these snapshots.
Then, the % load for each, is the incremental difference you just
found, versus the sum of the incremental differences of all of the
above.

It is important to do it this way. Initially, I thought the
incremental difference itself (as a number) should be something
meaningful. This is not the case. There is no way to measure the time
between any 2 snapshots in enough detail to get an expected total
incremental difference.

What is interesting is iowait.
To my understanding, iowait is not CPU load. The CPU is never in a
loop waiting I/O. This is a task for interrupts.
iowait is just 'idle' (the only difference is that during the iowait
time there were pending I/O operations - the CPU was idle though).

Costa

On Tue, Mar 15, 2016 at 1:18 AM, Rick Marshall <rjm at zenucom.com> wrote:
> If you look at the documentation you will see that the reported cpu load
> isn't really the load. Not really a problem in most cases but when looking
> for free cpu to run batch jobs the calculation has to be adjusted by the
> number of cores. Not sure if this affects you or indeed I'm not sure I know
> what %idle means on a multi-core system. eg 4 cores = 400% available less
> reported core usage = say 280% available. Divide by 4. Does that mean 70%
> available? Maybe.
>
> I have deeper scheduling concerns on heavily loaded systems that the kernel
> seems to somehow handle.
>
> If I get time I will try to look closer at my main concern which is data
> bottlenecks into the chips and the affect on performance. Is the cpu idle
> because it can't get data from the disks? can't get data through it's
> databus? fighting cache coherence?
>
> Just to give you an idea of what I try to work out for performance issues.
>
>
> On 15/03/16 10:10, Costa Tsaousis wrote:
>
> Hi Rick,
>
> Node.js is there. Check the wiki and the node.d directory in the source. I
> have already implemented the core and 2 plugins in it, one being the SNMP
> plugin.
>
> Regarding the CPU load, I am not sure I understand. Can you elaborate a bit?
>
> Thanks,
>
> Costa
>
> 15 Μαρ 2016, 00:41, ο/η Rick Marshall <rjm at zenucom.com> έγραψε:
>
> Hi Costa
>
> I have it running nearly constantly on my main server - the rest are a bit
> transient atm, but separate problem.
>
> Love it and working at building a plugin for Unibase so that I can monitor
> my database laods.
>
> Question: cpu load is poorly implemented in Linux in that multi-core
> machines (everything these days) don't report overall cpu load and if you
> have tried to configure batch you will have experienced the pain. Have you
> allowed for the multi-core machines in the cpu load?
>
> Thanks again - look forward to the node.js integration.
>
> On 15/03/16 09:28, Tsaousis, Costa wrote:
> Hi again,
>
> I need some feedback guys.
>
> Are you using netdata regularly?
>
> What is the most important thing you miss?
>
> To my view it is stable and pretty fast (although, I don't like the
> shell scripted plugins which I plan to move to node.js - they increase
> the total CPU consumption of netdata a lot).
>
> This is my short term TODO list:
>
> 1. Improve the web UI a bit (today I realized there is no way to zoom
> in/out the charts using the touchpad)
>
> 2. Work on its memory footprint to allow longer durations (probably
> split it in a short in-ram db and a longer on disk compressed db)
>
> What do you think?
>
> I know there are many more to do.
> But my primary goal is to have a stable and usable environment to build
> upon.
>
> So, do you use it?
> Are you happy with it?
> Is there anyone among us that has installed it on all his/her servers?
> (I have, of course :)
>
> Thanks,
>
> Costa
>
>
> On Sat, Feb 13, 2016 at 12:27 AM, Rick Marshall <rjm at zenucom.com> wrote:
> Thanks Costa. I'll keep you posted.
>
>
> On 12/02/16 19:23, Tsaousis, Costa wrote:
>
> Also,
>
> the dashboard supports documentation per chart. There we could write
> information about interpreting each chart, add links to external web sites,
> etc.
> If anyone is willing to help, please open a github issue to give you
> instructions on how to commit such info.
>
> Netdata is able to get system information for a lot more things that is
> currently presenting. In the source, I have commented all the code that
> collects data I don't know its value. Check this for example:
> https://github.com/firehol/netdata/blob/master/src/proc_vmstat.c
> It have written the code to collect 100 metrics, but only 5 are enabled. If
> you know the importance of the rest, please open github issues too.
>
> Costa
>
>
> On Fri, Feb 12, 2016 at 10:00 AM, Tsaousis, Costa <costa at tsaousis.gr> wrote:
> Thanks!
>
> 1. all the sections on the wiki have been implemented, but the empty ones
> are undocumented... Check your /var/log/netdata/error.log (it should be
> complaining about nginx and all the other modules that cannot automatically
> detect their config).
>
> 2. redhat/fedora, unfortunately no. I have installed it on debian/ubuntu,
> arch, gentoo. If you can help, please open github issues.
>
> Costa
>
>
> On Fri, Feb 12, 2016 at 6:16 AM, Rick Marshall <rjm at zenucom.com> wrote:
> You should be proud of it.
>
> Q: redhat/fedora support. Wiki has debian/ubuntu instructions. Have you
> installed on redhat/fedora?
>
> If not I will build an send results.
>
>
> On 12/02/16 10:42, Tsaousis, Costa wrote:
> Hi all,
>
> Have you checked netdata lately?
>
> Well, I am really proud for it...
>
> http://netdata.firehol.org/
>
> Check its wiki too (work in progress):
> https://github.com/firehol/netdata/wiki
>
> Costa
> _______________________________________________
> Firehol-support mailing list
> Firehol-support at lists.firehol.org
> http://lists.firehol.org/mailman/listinfo/firehol-support
>
> --
> *Rick Marshall*
> Director
> Zenucom Pty Ltd
> +61 411 287530
> www.zenucom.com <http://www.zenucom.com>
>
> _______________________________________________
> Firehol-support mailing list
> Firehol-support at lists.firehol.org
> http://lists.firehol.org/mailman/listinfo/firehol-support
>
> --
> Rick Marshall
> Director
> Zenucom Pty Ltd
> +61 411 287530
> www.zenucom.com
>
> _______________________________________________
> Firehol-support mailing list
> Firehol-support at lists.firehol.org
> http://lists.firehol.org/mailman/listinfo/firehol-support
>
> --
> *Rick Marshall*
> Director
> Zenucom Pty Ltd
> +61 411 287530
> www.zenucom.com <http://www.zenucom.com>
> _______________________________________________
> Firehol-support mailing list
> Firehol-support at lists.firehol.org
> http://lists.firehol.org/mailman/listinfo/firehol-support
>
>
> --
> Rick Marshall
> Director
> Zenucom Pty Ltd
> +61 411 287530
> www.zenucom.com