• BinkP Utility - Stats Generator

    From Jeff Earle@1:229/700 to All on Mon May 10 04:35:46 2021
    Hello all,
    Is there a Mystic Binkp Stats Generator Utility that would create a bulletin
    or message that would show volume of messages, files and posts, etc, for the system?

    I have looked at every Mystic source for utilities that I can find and came
    up empty. I dont see anything in the Mystic docs for MUtil to create one.

    Maybe I'm not looking in the right place.

    Thanks
    --- Mystic BBS v1.12 A46 2020/08/26 (Windows/64)
    * Origin: Mystic Realms - A Social Media Alternative (1:229/700)
  • From g00r00@1:129/215 to Jeff Earle on Mon May 10 12:10:56 2021
    Hello all,
    Is there a Mystic Binkp Stats Generator Utility that would create a bulletin or message that would show volume of messages, files and posts, etc, for the system?

    I have looked at every Mystic source for utilities that I can find and came up empty. I dont see anything in the Mystic docs for MUtil to
    create one.

    I think someone made some log parsers but I haven't tried them. I am not sure where to look to find them.

    Mystic keeps stats itself for nodes but it does so with packet size not number of messages (so its sourced from the BINKP protocol not message tossing). If you want some message volume by message area then there are several JAM utilities that can generate those types of reports but it would not be specific to nodes.

    Do you have some examples of what you might like to see? Maybe I can built somethinhg to close any gaps between what is
    available and what isn't.

    --- Mystic BBS v1.12 A47 2021/05/02 (Windows/64)
    * Origin: Sector 7 | Mystic WHQ (1:129/215)
  • From Jeff Earle@1:229/700 to g00r00 on Mon May 10 13:30:18 2021
    Is there a Mystic Binkp Stats Generator Utility that would create a bulletin or message that would show volume of messages, files and pos etc, for the system?
    Do you have some examples of what you might like to see? Maybe I can built somethinhg to close any gaps between what is
    available and what isn't.

    Something like this would be helpful:

    Area In msgs Out msgs Bad Dupe In bytes Out bytes

    ALL-POLITICS - 0.0% - 0.0% - - -- 0.0% -- 0.0% ALLFIX_FILE 24 5.3% 35 1.1% - 9 68k 6.2% 73k 0.7% ALLFIX_HELP - 0.0% - 0.0% - - -- 0.0% -- 0.0% ALT-BBS-ADS 1 0.2% 1 0.0% - 1 1.5k 0.1% 1.5k 0.0% ALT-BBS-MISC - 0.0% - 0.0% - - -- 0.0% -- 0.0%

    And maybe another showing the In/Out as above but by each Net's traffic.

    Yes I was incorrect in assuming it would be from BinkP, MUTIL that tosses messages and TICs would have to produce it.

    I see it as the 1st line of defense of something that may be wrong and point
    to where to look in the logs instead of having to comb through them.

    If you need a tester for an implementation as part of MysticBBS just let me know.

    Thanks
    --- Mystic BBS v1.12 A46 2020/08/26 (Windows/64)
    * Origin: Mystic Realms - A Social Media Alternative (1:229/700)
  • From g00r00@1:129/215 to Jeff Earle on Mon May 10 15:10:18 2021
    Something like this would be helpful:

    Area In msgs Out msgs Bad Dupe In bytes Out

    There are some generic open sourced JAM utilities that produce reports like this but it would't include the dupe stuff and I can't for the life of me remember where to find any of them.

    There may be one or two people who made similar things specifically for Mystic too, but again I don't know what those are I just remember hearing things.

    I can see the IN/OUT messages on a per-base level being a good thing to track and right now it has to come from log parsing MUTIL, so that is one place for some enhancements. But even if that is tracked internally that still doesn't tell you anything unless you specifically go look. Maybe what we're really looking for could be some automated notifications when certain things happen?

    I see it as the 1st line of defense of something that may be wrong and point to where to look in the logs instead of having to comb through
    them.

    If error tracking is your end game then there are some things you may not be aware of, although they are more designed to be
    hub-based automation features. Mystic does track things like crash errors, in/out files and bytes via BINKP. You can see all of these stats for each node in the EchoNode Editor whenever you want...

    There is also the "Echo Node Tracker" which will handle all sorts of automated tasks for hubs like resetting stats on time-based intervals, automatically deactivating nodes after periods of inactivity, unsubscribing msg/file bases after automated deactivation, purging outbound packets after inactivity period, automatically swapping from "crash" to hold-type mail delivery after a certain
    number of days of errors or overall error threshhold, etc.

    But again all of that either happens automatically or requires you to make the effort to go look yourself. Perhaps we need to build in notifications?

    Open to any ideas of course, if you (or anyone else) has them.

    --- Mystic BBS v1.12 A47 2021/05/10 (Windows/32)
    * Origin: Sector 7 | Mystic WHQ (1:129/215)
  • From Jeff Earle@1:229/700 to g00r00 on Tue May 11 10:05:04 2021
    There are some generic open sourced JAM utilities that produce reports like this but it would't include the dupe stuff and I can't for the life of me remember where to find any of them.

    I have searched as well and found nothing. I will keep looking

    There may be one or two people who made similar things specifically for Mystic too, but again I don't know what those are I just remember
    hearing things.


    I've been to all the Mystic related sites I can find and none seem to have anything for stats on message bases, everything else though!

    I can see the IN/OUT messages on a per-base level being a good thing to track and right now it has to come from log parsing MUTIL, so that is
    one place for some enhancements. But even if that is tracked internally that still doesn't tell you anything unless you specifically go look. Maybe what we're really looking for could be some automated
    notifications when certain things happen?


    I hear you on going to look. I prefer to get a report of what the system is doing and if I see something in the report, yes, go look. But with a large amount of message bases and networks, its too much to look though the log for potential problems. Maybe its the reports I am used to in my job that brings this to mind. With all the telemetry that gets recorded for our fleet of trucks, I simply can't just go look for issues. I need those reports to point me to potential issues so I know where to look.

    Had an issue the other day with 1 hub and if I didn't happen to be scrolling through the message bases I would never have found it as I don't read that peticular base. Saw that an enormous amount of messages were arriving each
    poll to the hub. I am talking 10's of thousands of messages. Investigated an found messages coming in decades old. Emailed the sysop right away and the problem was fixed. Fixed in 1 day. That was just by luck. Reading a report everyday would help so much.
    I would be very interested in 'automating' these internal reports you
    touched on. They would be very helpful.

    If error tracking is your end game then there are some things you may
    not be aware of, although they are more designed to be
    hub-based automation features. Mystic does track things like crash errors, in/out files and bytes via BINKP. You can see all of these stats for each node in the EchoNode Editor whenever you want...


    I will check this out. No, I was not aware. After I look, I may have some ideas.

    There is also the "Echo Node Tracker" which will handle all sorts of automated tasks for hubs like resetting stats on time-based intervals, automatically deactivating nodes after periods of inactivity, unsubscribing msg/file bases after automated deactivation, purging outbound packets after inactivity period,


    I can see where Hubs would use this. Gives me more to look at.:-)

    But again all of that either happens automatically or requires you to
    make the effort to go look yourself. Perhaps we need to build in notifications?

    Open to any ideas of course, if you (or anyone else) has them.

    If you don't mind, I do have some ideas that would fit right in with built in notifications.

    I have that in my work and my Home streaming server. It notifies when
    something isn't working and reports what was happening just before the issue.
    I can correct immediately (remotely if needed) to get the streaming server
    back online for the family viewers. Better that getting a call from daughter who just says "Its not working and can't watch my show".

    Thank you for the information. It is very helpful.
    --- Mystic BBS v1.12 A46 2020/08/26 (Windows/64)
    * Origin: Mystic Realms - A Social Media Alternative (1:229/700)
  • From g00r00@1:129/215 to Jeff Earle on Wed May 12 15:54:40 2021
    I have searched as well and found nothing. I will keep looking

    I did some basic searching too and its sad that some of these things are getting lost.

    I'm surprised no one else has jumped in to tell us of them too since I know others have used them in the past.

    large amount of message bases and networks, its too much to look though the log for potential problems. Maybe its the reports I am used to in my job that brings this to mind. With all the telemetry that gets recorded for our fleet of trucks, I simply can't just go look for issues. I need those reports to point me to potential issues so I know where to look.

    For most situations you're not going to need to go through logs but in that situation you described with the old messages yeah that would slip through the cracks. We need to figure out a way to identify that scenario easier.

    When you go into Echomail nodes you can see the number of days nodes have been inactive, if there is a "streak" of crash errors when attempting to send mail (and how many), the last time data was successfully sent, the last time data was successfully received, the inbound flow byte size and file count, the outbound flow byte size and file count...

    You can see this all reported in real time without needing to generate anything. I think the issue is that just like you'd be required to generate and review a report, you have to go in and look. So maybe being able to set parameters that allow you to be notified would be a good first step.

    If you don't mind, I do have some ideas that would fit right in with
    built in notifications.

    Absolutely I would love to hear it!

    --- Mystic BBS v1.12 A47 2021/05/11 (Windows/32)
    * Origin: Sector 7 | Mystic WHQ (1:129/215)