MmStats: Delicious Data

Documentation | Package | Code

About

Mmstats is a way to expose and read diagnostic values and metrics for applications.

Think of mmstats as /proc for your application and the readers as procps utilities.

This project is a Python implementation, but compatible implementations can be made in any language (see Goals).

Discuss at https://groups.google.com/group/python-introspection

Goals

• Separate publishing/writing from consuming/reading tools
• Platform/language independent (a Java writer can be read by a Python tool)
• Predictable performance impact for writers via:
  • No locks (1 writer per thread)
  • No syscalls (after instantiation)
  • All in userspace
  • Reading has no impact on writers
• Optional persistent (writer can sync anytime)
• 1-way (Publish/consume only; mmstats are not management extensions)

Usage

Requirements

CPython 2.6 or 2.7 (Windows is untested)

PyPy (only tested in 1.7, should be faster in 1.8)

Using

1. python setup.py install
2. import mmstats
3. Create a subclass of mmstats.MmStats like:

class WebStats(mmstats.MmStats):
    status2xx = mmstats.CounterField(label='status.2XX')
    status3xx = mmstats.CounterField(label='status.3XX')
    status4xx = mmstats.CounterField(label='status.4XX')
    status5xx = mmstats.CounterField(label='status.5XX')
    last_hit = mmstats.DoubleField(label='timers.last_hit')

4. Instantiate it once per process: (instances are automatically thread local)

webstats = WebStats(label_prefix='web.stats.')

5. Record some data:

if response.status_code == 200:
    webstats.status2xx.inc()

webstats.last_hit = time.time()

6. Run slurpstats to read it
7. Run mmash to create a web interface for stats
8. Run pollstats -p web.stats.status 2XX,3XX,4XX,5XX /tmp/mmstats-* for a vmstat/dstat like view.
9. Did a process die unexpectedly and leave around a stale mmstat file? cleanstats /path/to/mmstat/files will check to see which files are stale and remove them.

Further topics

• API
  • Models
  • Model Fields
  • Reader API
  • Internal Defaults
  • mmap Compatibility Wrapper
• Development
  • Getting Started
  • TODO
  • mmap Format
  • Scrapped Ideas
• History
  • 0.7.0 “Local Artisanal Stats” released 2012-10-02
  • 0.6.2 “Graphtastic” released 2012-03-23
  • 0.6.1 “MANIFEST.out” released 2012-03-08
  • 0.6.0 “PyCon 2012” released 2012-03-08
  • 0.5.0 “100% More Average” released 2012-02-25
  • 0.4.1 “Derpstats” released 2012-01-31
  • 0.4.0 “On the Road to Pycon” released 2012-01-17
  • 0.3.12 “Meow” released 2011-11-29
  • 0.3.11 “Rawr” released 2011-11-29
  • 0.3.10 “π²” released 2011-11-28
  • 0.3.9 “MLIT” released 1970-01-01
  • 0.3.8 “Hapiness” released 2011-11-20
  • 0.3.7 “Depressive Realism is for Winners” released 2011-11-17
  • 0.3.6 “The M is for Mongo” released 2011-11-09
  • 0.3.5 “Ornery Orangutan” released 2011-10-20
• Contributors
  • Developers
  • Reviewers

Indices and tables

• Index
• Module Index
• Search Page