This is not where I keep my stuff

I keep my stuff here

The Breakdown of the Tribe and Its Anticipated Rebirth

community :: society

A good friend who has very little contact with the open source
movement asked me to offer ideas that could assist a non-technical
organziation for which we have both volunteered much time and energy.

As a result, I summarized (at length) ideas that other friends of
mine and I have been discussing. Though focus here is put on the open
source community, our deepest concern has been for society as a whole
and useful models of which small groups can take advantage and from
which they can learn.

I've been doing some serious analysis concerning societal paradigms,
especially in regard to evolving definitions of "tribe" over time.
Related, is the issue of group dynamics and group health. The
efficiencies and "health" of the massive open source community has been
especially puzzling until recently.

Let me start with background:

It is my current view that tribes existed as cohesive units due to
violent "external" conditions (including predators, enemies, and other
hinderances to survival). If the forces at play were external only,
we'd still have tribes today. However, I think there was a balance of
force at play: external threat vs. internal mutual dependance. Tribes
formed to increase chances of survival; they stayed together because,
even though others are a pain in the ass, they needed the lecherous
cobler, the violent warrior, the reclusive black smith, etc.

So why did the tribes start falling apart? I think it was
technology... in the form of roads, improved communication, better
farming techniques, whatever. With improved technology, comes increased
independence of individuals, greater power of each person to do things
on their own and be less dependent. Each advance in technology has
further empowered those capable of wielding it.

Technology also tends to abstract things. For instance, the modern
grocery store is an abstraction of a much longer, real-world process:
tilling, planting, harvesting, going to market. These things still
occur, but just not in the consciousness of the modern consumer... the
process has been replaced with a 5 minute zip down the road in an iron
"horse."

Furthermore, it is my opinion that the extended family is an eroded
form of the tribe, and the nuclear family an even further decay. This
has concerned me for some time, especially if you believe that society
tends to represent lower common denominators (I'm thinking of average
level of consciousness). These are related in that if group
consciousness is in the process of evolving, and we have "just now" (in
the past 2000 years) experienced a dissociation of the tribe, then we
are very far from evolving a healthy replacement. It could be anywhere
from several decades to hundreds of years before we see a large-scale
integration of whatever form the next healthy "tribe" will take.

However, like so many other things in nature that evolve slowly over
time, we may actually be right in the middle of it. More digression
first:

We drive on superhighways, lights cover the midnight city like a
spaceship... or an artist's rendition of electrons in a computer chip.
We can send people up into space... where they actually experience
relativistic changes in time (minute, but measurable). This is a world
of science fiction... become fact. In the 30s, 40s, and 50s, wacko
writers were envisioning crazy worlds that didn't do half the stuff we
can do now. One of those wackos was Asimov, whose early "predictions"
for the "far future" were outstripped within 20 years of some of his
short stories.

Our world is not the one our kind has known for millennia. It's
changing at every level. It was a crushing blow to me personally, when
this really hit home: there is no going back, there is only accepting
what we have become and making this work for us... us homo... what?
perhaps homo connectivus.

All this change sneaks up on us, though. Somehow we still function
as though parts of our minds and being were in the mid-1800s... or
800s... or 2800 B.C.E. With our changing world and the increased amount
of abstraction we are forced to deal with, I think we tend to seek safe
harbor in the human past. However, with each generation, artists,
poets, and musicians have been egging us onwards. Trying to tune our
ears, eyes, hearts, and minds to the demands that will be increasingly
made upon us and future generations.

Which brings me closer to the point of this whole thing. At the
beginning, I said how I think I am beginning to understand the open
source movement, a movement that, as one of its mottos, says "Use the
Source, Luke." The only way I was able to start getting it was through
abstraction.

Now mind you, open source was condemned to failure by some of the
biggest business minds in the world. It has baffled people for years:
how can it continue to exist? How does it grow? There is no money, what
is the economic model for this headless behemoth?

The secret of the open source community is that it works like a
brain. There is no ultimate hierarchy in a collection of billions of
neurons (perhaps an emerging pre-consciousness though?). More
importantly, it doesn't matter how many sick, unhealthy cells there are
in it (within obvious limits). If a set of neurons that were a major
conduit for certain signals break down, the signals get rerouted. The
brain is a highly redundant, fault-tolerant network.

The open source community is composed of millions of individuals all
over the globe. I work on projects with team members having multiple
PhDs, members who are university students, high school students and
middle schoolers, musicians, mathematicians, psychologists, blue-colors
workers, etc. The community is becoming all-pervasive. It is one of the
most tolerant, accepting, healthy and helpful communities I have ever
been involved in.

But that didn't make sense, when I thought about it. I *know* some
of those people, and many are not healthy, and some are down-right
mean. After thinking about it, I realized that the best way to consider
it was via an abstraction: Graph Theory from mathematics. Graph Theory
has nothing to do with pictures: many graph theoretic problems focus on
network connections and traversals. Weighted values. Navigating through
a "problem space" with the least expenditure of energy. Companies like
FedEx use graph theory calculations to determine the
fastest/cheapest/most productive delivery routes.

Communities with lots of members form vast networks. People are
nodes on the network and any possible means of interacting with those
people are the connections between the nodes. If someone in the open
source community treats me like shit, I don't get bent our of shape. I
make a mental note, and maybe tell my friends what a dick that person
is (thus giving that node a negative weight with less chance of
traversal). But I then try to find an alternate path to the information
I need or the information I need to share. Over time, I discover
faster, more efficient, path ways of interaction that improve my
quality of life while interacting with other members of the community.

This may seem like a cold distillation, but I am not trying to
capture or limit anything. I'm just trying to find accurate or useful
*approximations* of behavior. In highly trafficked networks, patterns
of usage begin to emerge. In networks of humans and their projects,
mini-networks tend to emerge in areas of higher-than-nornal usage.
These, mini-networks are little communities... or, shall we say,
"tribes". They are dynamic in nature and tend to develop patterns of
internal flux (immigration, emigration) as well as movement/migration
of the community/tribe as a whole. These communities often don't have
set leaders. However, the ones without leaders have very well-defined
goals and mechanisms for determining consensus. The ones with leaders
tend to have a very democratic approach to their leadership. The most
effective communities exchange an extraordinary amount of information:
constant communication and feedback is critical to success.

In networks, there is no "ultimate dependance": people and lines of
communication (nodes and connections) are only used to their abilities.
There is no round-hole-square-pegging -- that just takes too much time.
In public communication, people observe what others say and how they
say it. From that, they make initial guesses at who is good at what.
Initial guess plus trial and error. Sounds like differential
equations... and artificial intelligence ;-) You can have complete
losers, slackers, etc., in any group. They tend to add flavor ;-) They
just don't end up being used as part of the network when stuff has to
get done. They get used for whatever they are good at... like making us
laugh or letting us vent.

Once I saw this, it became clear how this could be a replacement for
the tribe. First a recap. Tribes (maybe) fell apart for the dual
reasons of:

1) increased chances of survival outside the structure of the tribe, and


2) increased convenience of obtaining amenities without having to depend on tribe members.

Since then, we've been wandering about, trying to rediscover the tribe,
trying to reinvent that which has already become extinct for the simple
fact that the causes and conditions no longer exist. Yet we evolved
with the need for the tribe; its lack is therefore part of the modern
human experience of longing.

As our educated population becomes increasingly more so, we are
finding it more and more difficult to be lead by people who represent
the lower common denominators. Perhaps not so much for what they are,
as what they represent: an outmoded form of governance. Intelligent
networks don't have to tell each node what to do. They just need a
purpose and the resources to accomplish their goals.

The need for Survival has evolved into the need for improved Quality
Of Life. In the same way the amenities support survival, Joy supports
Quality of Life. So, tribes may be able to reform under the conditions
that members experience improved quality of life and joy in their daily
activities. Joy is an amazing antidote to daily human annoyances. Joy
can bind groups with a common vision together where nothing else would
have.

The people working on open source projects do so out of love and
joy. We love what we do. Those of us that are truly fortunate get to
work on open source *as* our jobs. The rest, though, do their jobs, and
when off work, they increase their quality of life by working on that
which brings them joy... and they do so in an open, non-hierarchical
system, with excellent communication, working with others to accomplish
things that could not be done by one, lone person.

Star Wars

entertainment

There has been some complaining about the changes George Lucas
has made to the new DVD release of StarWars. And you know what? I don't
care! The spirit of it remains in tact, and unless you're the Comic
Book Guy from the Simpons, the minute changes that have been made
shouldn't matter to you.

I'm watching the DVD of A New Hope right now, and holy shit -- it's
the first time I feel like I'm back in the San Diego theater as a
little kid, watching it for the first time. Awseome!

Thanks George!

We love you!

Modeling Human Behavior

In my recent post about generating personality types with python, I discussed a partial implementation of a Myers-Briggs module, using linear algebra to perform various operations on vectors that represent one's "Jungian" temperament.

I have finished all of the basic code and the module can be used to return the best matches for any given temperament. However, I would like to be able to provide a calculable method for a sliding-scale of compatibility. This could be used in games to simulate interactions between a player and an NPC. In a room full of NPCs, it could be used to establish flows of communication, charisma gradients throughout the room... it could be used to augment 2-dimensional random walks of NPCs (dispersion/Brownian Motion) to preferences for certain gradients established by field lines of compatibility between individuals.

Time to break out the books and dig back into multivariate calculus...

These thoughts arose after I wrote a simple implementation of personality.generator.BasePerson. That made it clear that the interaction of instantiated person objects is not too far away (conceptually; programatically, there will most likely involve a great deal of code and time).

This was the catalyst to start investigating what research has been done on the modeling of human behavior...

The first paper I read was a true gold-mine: "Human Behavior Models for Game-Theoretic Agents: Case of Crowd Tipping." It sent me off on tangents into alluring, siren-like world of http://mathworld.wolfram.com/ and http://scienceworld.wolfram.com/physics/. It also referenced other excellent papers I was able to find online. Below I have listed some of the interesting papers I came across as well as some excerpts.

Human Behavior Models for Game-Theoretic Agents: Case of Crowd Tipping (http://tinyurl.com/7xpqq)

• Their emotional system uses the OCC model (originally formulated by Ortony, Clore and Collins, an emotional appraisal model).
• Their cognitive framework is a modifed MDP (Markov Decision Process) that incorporates a BDI model (Belief-Desire-Intention).
• They have a stress and physiological subsystem that "initially reacts to a set of stimuli that are perceived from and/or experienced in the environment." They "model eight physiological reservoirs or stressors, including: energy, sleep, nutrients, noise and light impacts, and other physical capacities..."
• They discuss a simple motor subsystem for interaction in a microworld.

More Realistic Human Behavior Models for Agents in Virtual Worlds (http://tinyurl.com/5fy6p)

"There are a number of specific agent research issues which arise such as, among others, building and deploying agents that can:

• Acquire and maintain knowledge of the environment, of other agents actions (human or synthetic), and of their own experiences, successes and failures.
• Create tactical plans and carry them out in a believable manner covering both reactive and deliberative behaviors in the presence of other players.
• React appropriately to stress, fatigue, and anxiety and reflect their integrative impact on judgment and performance...
• Construe emotional reactions and mood as stimuli to personal behavior and choice...
• Cope with multi-stage activities (e.g.,campaigns), strategic plans, and survival and decide when to make tactical sacrifices (or not) for the betterment of larger contexts (meta-games)..."

Excellent comment, and one of my biggest problems with the current state of games:

"The coding of the software by game-makers and military wargame modelers is usually created in the simplest manner and in response to a market or project deadline. Historically there has been little capability for deliberative reasoning by these agents and one doesn't encounter theoretically rigorous constructs that can be counted on to perform according to mathematical theory. Due to these constraints, perhaps the most popular construct for game creation and other simulations is the finite state machine (FSM) approach. FSMs offer the rudiments needed to implement Markov chains and MDPs, and to organize agents into iterative meta game-playing participants within a multi-stage, hierarchical network. However, the vast majority of FSM systems implemented to date are programmed from the bottom up, with little agent reasoning and without the concept of a larger theory to validate them against."

"Most of the games and artificial lifeforms out there are artistically and stylistically impressive (very impressive), but not entirely faithful to real human behavior. Usually, the game makers hire a psychologist to verify the behaviors seem plausible, but they rarely get down to actual fine-grained details and rarely implement models based on first principles (e.g., reflexes, reaction times, effects of stress, fatigue and adrenalin effects, judgment rates, etc.)."

And this puts a fine point on game theoretic issues:

"Traditional game theory is mathematically rigorous but overly simplistic. The games that are most commonly studied involve fixed, highly constrained payoff tables; intellectually hobbled opponents; and single layer of play where metagaming and systemic thinking is not allowed."

I could just go on and on about this paper! I highly recommend reading it.

Integrating the OCC Model of Emotions in Embodied Characters (http://tinyurl.com/67xcr)

This paper has a fantastic flow chart of the original OCC model; it's a perfect start for programmers. The author discusses five phases of emotional processing: Classification, Quantification, Interaction, Mapping, Expression.

A Model for Personality and Emotion Simulation (http://tinyurl.com/42qf5)

From the abstract:

"This paper describes a generic model for personality, mood and emotion simulation for conversational virtual humans."

• Discusses mathematics for PE model (Personality and Emotional State) as well as PME model (Personality, Emotional State, and Mood).
• Discuss OCEAN model: Openness, Conscientiousness, Extraversion, Agreeableness, Neuroticism.
  

Conversational Agents for Game-Like Virtual Environments (http://tinyurl.com/44yrg)

This paper has a well-developed architecture for its conversational agent software. I will definitely be adapting their insights...

A Reliable Computational Model For BDI Agents (http://tinyurl.com/5bzvj)

From the Introduction:

"The BDI approach is based on the study of mental attitudes... and tackles the problems arising when trying to use traditional planning in situations requiring real-time reactivity. The Beliefs represent the informational state of a BDI agent, that is, what it knows about itself and the world. Desires or goals are its motivational state, that is, what the agent is trying to achieve. A typical BDI agent has a so-called procedural knowledge constituted by a set of Plans which
define sequences of actions and tests (steps) to be performed to achieve a certain goal or react to a specific situation. The Intentions represent the eliberative state of the agent, that is, which plans the agent has chosen for eventual execution."

In the abstract and later in the paper, they mention the "...many problems concerning concurrency control and recoverability..." and this made me think of Twisted (python) as a possible framework to address these issues. Twisted was a result of what is now called Imagination, a framework for building playable virtual realities. I haven't seen them
focus on behavior modeling, but may well have kept such things in mind.

Reading about Markov/BDI agents got me thinking, so I went to the wolfram math site. I started at the Random walk page:

• http://mathworld.wolfram.com/RandomWalk.html
• http://mathworld.wolfram.com/RandomWalk2-Dimensional.html
  

then the Brownian Motion page (possible inspiration for NPC movement in a crowd):

• http://scienceworld.wolfram.com/physics/BrownianMotion.html
  

then to Percolation Theory from fluid dynamics (Greg! we're getting close to your expertise here! possible inspiration for movement of memes, moods, etc., in a crowd of NPCs)

• http://mathworld.wolfram.com/PercolationTheory.html

These two don't mean much to me, as I have no idea how to implement them practically. I'll have to find some source code that explores them:

• http://mathworld.wolfram.com/MarkovSequence.html
• http://mathworld.wolfram.com/MarkovChain.html
  

Technorati Tags: software, games, mathematics, ai, python

Python, Linear Algebra, and Character Development in Games

So, a few things:

1. I was bored last night working on the AOL project (QA phase... blech!)
2. I therefore played some NWN with my new level 15 samurai (12 fighter, 3 weapons master prestige class).
   
3. I wanted to relax after that, so I played some simcity4 and ran my city into serious backruptcy.
4. Then I wanted to challenge my mind :-) So, back to python!

Though not an avid player (too little time and too few friends into old-school RPGing), my favorite part of D&D is character development. I absolutely love it. As a by-product of this, I would also like to see much greater complexity of interaction and response་between players and NPCs. Thus, a while ago I decided to start thinking་about that, and one of the results was the seasonal system for Rook:

Rook Seasons and Planets

and then system of astrology used on Rook:

Rook Astrology

Categorizing that stuff and breaking it down for compatibilities and་temperments is going to be quite a task. However, it will be a fun one.

I think a cool side affect of writing that code is that we will then be་able to translate between Rook signs and Earth signs, and thus we could་actually "use it" or at the very least, refer to it meaningfully. (More་than just I'm an Igou, Karthik is a Kvuang, and Greg is a Rookwuni.
Heh, that's still pretty cool.)

The code I did write:

Last night, I wrote the first code. I decided to start with something་known and easily classifiable, temperaments as well as compatibilities:་Myers-Briggs. The initial delvings are now in the subversion repository.

This stuff is really cool, because I decided to use some tricks from་quantum mechanics math (the stuff we did with eigenvectors/values,་etc.) and use matrices and vectors composed of signed values (+/-1).་Under certain operations, patterns begin to emerge (nullifications and
doublings). I haven't explored this fully yet, but the dot products་with one's personality vecotor and a compatible type seem to result in་'2'. I have to exhaustively show all dot products to really establish a་pattern. But at the very least, this is interesting information and
methodology.

The fact that it may be reducible to operations in linear algebra is་good! Computable = predictable = appropriate system for gaming and་computations.

Technorati Tags: software, games, mathematics, python

The File System as a Database and the Last Post of Summer

software :: python

Heh, that title reads like the title of a Victorian-era
pseduo-science journal article. However, I really do feel that August
31st is the last day of Summer... that come September first, Everything
Changes. But not in a bad way -- Fall is amazing. It's exhilarating in
a funny, erie kind of way.

So, this is the last blog entry of the Summer, 'nough said.

As for the rest of the title, I've been discussing this topic with
friends of late. Basically, how can we access the file system in
python, treat it like a database, search on it like a database, and
write code for it in such a way that when we are ready, we can
migratate to a database with no code changes (only config or module
import changes)?

Well, we've debated the issue(s) back and forth. I even asked
Phillip Eby about it on the PEAK mail list (we will be using PEAK for
this project... once we learn it!). But I think we've all been trying
to make the problem and solution too general. A very simple and rigid
API would work for us right now. It means less time spent on R&D,
and since this is for a paying customer, that means more money for us
in the long run.... as long as what we implement leaves enough
flexibility for future change.

So, without further ado, an adaptation from a post to the mail list today:

If we have a file on the filesystem, then the full path + the file
name uniquely identifies that file. In my limited knowledge of OODBs,
this is pretty standard (path-to-object = UID). Then there's the file
itself, which contains some data. Additionally, however, is the path:
it contains data that is just as important as the data inside the file.

UID: full path + filename

Data: stored in file at /fullpath/filename

Data: stored in path and filename

How do we think about this problem? If this were a table, we might be looking at a schema like this:

Table

-----

id: full_path + filename

blob: rrd file/text file/ini file/xml file with DTD/whatever

additional field 1:

additional field 2:

additional field 3:

...

additional field n:

I'm not proposing an OR mapping here: that's complicated shit. Way
beyond me. Some of the biggest brains in the software development world
are working on ways to do that which make sense and work right.

I'm just talking about doing something simple and straight-forward.
Something that's easy to configure and easy to migrate from a
filesystem to an RDBMS.

This shouldn't be as hard as I was thinking originally. The only
issue is that for every implementation, there will need to be a
configuration. This is because, by their nature, database tables and
fields defined therein are fixed; directory structures aren't/don't
have to be. A configuration would "lock" a directory structure... you'd
have to have an API (or something) that defined what each level of the
directory structure indicated, as these would have to be mappable to
fields defined in a table (for migration to a SQL framework).

Additionally, if you wanted to move the data stored in you blob
field out of its own little format into SQL, you'd have to define an
additional config/API for mapping its data to more fields in the
table...

So what you'd really have here is a directory structure schema and
then a file storage schema. Using the two together, you'd get what I
originally asked Phillip Eby about...

PEAK already has 'peak.storage.files' which lets you interact with
text files transactionally. We could do something like this for other
types of data-containing files at the end of whatever directory tree.
The combination of this with an implementation of a queryable filsystem
data interface should leave us with a fairly powerful tool for many of
our projects.

Questions to ask:

* What constitutes a database? (root dir and below?)

* What constitutes a table? (all directories at the first level, inside the root dir?)

* What constitutes a row? (every branch from root? this means all paths
from the root dir have to have the same number of dirs, subdirs, etc.)

* Can there be no file at the end of a path?

* Can there be empty files at the end of a path?

* Can there be multiple files at the end of a path?

* Can there be files in intermediate directories? (dirs that aren't the end of a path; good place for metadata?)

Freemind

software

It's been a couple years since I checked out freemind (a free "mind mapping" software for oganizing text/notes in loose relationships), and I just downloaded the latest version for Mac OS X. I must say, I'm impressed.

The software is very stable, provides *exactly* what I need, and is free :-) I highly recommend interested parties check it out:

http://freemind.sourceforge.net/.

Now, if they could only develop a live wiki version of the same thing ;-) Well, it's java, so I imagine it wouldn't be too hard to turn this into a servelet... of course, I'm not a java programmer and I have no right to make that statement!

Python port, anyone?

PyHTMLWidgets

python :: html :: web

Well, I've been working with some ancient code base (1995) to produce HTML for PyHTMLWidgets.... and thanks to a friend of mine (Paul Taney), I have realized that the proper tool has been in front of me the whole time: HTMLgen.

Once I am finished with the looming projects (two weeks, I hope!), I
will update the code base for PyHTMLWidgets and provide a download.
It's not rocket science... in fact, it's not science at all: it's
convenience. In the world of awesome templating systems and complete
tool sets for projects, there are customers out there that want things
in a particular way... sometimes, this precludes the use of some of the
better tools sets out there, and the need arises for something
completely independant and without dependancies on a larger system.

Thus, the ever-present need for separate tools with the ability to loosely bind them together...

Update:

A benchmark of different xml tools in python led me to the ElementTree web site and I am excited to see what this tool can do. There are a couple possibilities for PyHTMLWidgets:

* Use HTMLgen

* Use ElementTree instead


* Rewrite HTMLgen to use ElementTree

I'll have to spend more time thinking about potential features (mmm,
candy... mmm, scope creep) as well as running some benchmarks, but this
could actually be an interesting project...

Update:

Well, I had a chance to look at the source for HTMLgen, and it's
really not what I need. On the other hand, I've been playing with
ElementTree and that is a fantastic tool set. I am very pleased with
it's potential, and I think that it will provide *very* significant
increases in speed over what we are currently using.

My Dream Home Robot

technology

Two years ago, I had the desire to build a small flying "robot"
that I could control remotely and connect to an X10 home automation
setup. Turns out that I probably couldn't have done it at the size I
wanted, seeing how Seiko Epson has just produced it and it has the
smallest, lightest gyro sensor in the world. (See http://news.bbc.co.uk/1/hi/technology/3579232.stm.)
Hell, I probably couldn't have done it no matter what, seeing how I've
never built a robot in my life! To be useful for what I wanted, it had
to be wirelessly remoteable and capable of sending images (either
static snap shots or live video). This little critter meets such
requirements.

This is especially cool to me, far more cool than wheeled robots. It
receives messages via bluetooth, and with its micro camera, you could
remote it through your house, checking things like ovens, toasters,
irons, etc. It can operate for 3 hours before running out of juice in
its on-board batteries (rechargeable?). I would imagine that once you
had a device like this plugged up to your home automation system, new
ideas for useage would crop up by the hundreds...

! Not Blogging

technology

Yup, that's a double negative. I have no time to do it, so I
can't... and I'm not... not... blogging. In fact, I'm supposed to be
writing code for AOL...

Anyway, I think a friend of mine got mentioned in Clay Shirky's latest article
about Situated Software (of course, when I saw that title, all I could
think about was ATHF's Meatwad singing MC Peepants'/MC Chris's lyrics
"...get re-situated..."; yeah, I know I'm fucked ;-) ). Paul Barry is
my friend's name; he's working for NYU and taking classes. Really smart
dude. He built an LMS from the ground up for NYU. Unless my memory and
email addresses have failed me, Clay typo'ed Paul's name. Or it might
not be the same Paul... still waiting verification from Paul...