Thread synchronization

Chris Uppal coughed up:
> Thomas G. Marshall wrote:
>
>> It is not the object in total that is protected. It is a section of
>> code that you specifically force blockage into. If you instruct
>> people to think that synchronization is protecting objects, then you
>> are misleading them horribly. The bottom line is not that they are
>> objects. The bottom line is that a section of executable code is
>> protected from re-entrance by another thread.
>
> I disagree with this completely.

I read through this post, and I'm not sure that you do. Maybe "mostly".


> First off, it is not -- no matter what your stance on OO, or even
> whether you are using an OO language -- code that is being protected,
> it is /state/.

Sure, except that I can program *procedurally* in Java and still
synchronize, and protect particular lines of code. No objects. No
object-state. If you want to view the holder of the lock as maintaining a
"state" then go ahead, but it's not because it is OOP. It's more or less
only "holding" a mutex.


> There are one or more variables whose values must be
> "managed" to allow correct concurrent access. So at minimum it is
> misleading to talk of protecting code, you should be talking about
> protecting data.

Using that terminology, /all/ programming is merely data. In Prolog you'd
be close.


> Now in the OO world (and I'll accept, that Java is OO in this matter),
> data/state is normally bunched into semantically coherent units
> called objects. So the most natural thing to do is to manage
> concurrency at the level of the overall state of one object. In that
> case we are clearly using synchronization to protect the (state of)
> the object. Just as Eric says. And the normal way to express that
> is with code synchronized on 'this' -- synchronized methods for
> instance.

You can do that, but it isn't the bottom line. This is why I suggest to
newbies to create a separate object specifically for synchronization. I
posted why elsethread, but mostly to keep the instance vs. cl*** method
mistakes to a minimum.


> Now in some cases, the state that needs to be protected will be either
> distributed across more than one object, or be less than the entire
> state of an object. Such cases are rare -- as you'd expect because
> the object is (or should be) expressing a semantically coherent unit
> of state, and the protection will normally be expected to follow the
> same boundaries, precisely /because/ it is protecting semantic
> coherence. Still, such cases do occur, and in those cases, and
> /only/ in those cases, your more "advanced" technique of using a lock
> object is appropriate.
>
> BTW, I don't think "lock" is a good name for a lock object. For the
> reasons given above, it is misleading to use it to protect exactly
> the state of 'this'.

Sure, "lock" /is/ a little goofy. I suggest using a term apropos to the
algorithm. In other's code I often see "lock" (bad), or "{issue}Lock", or
"{issue}Mutex", such as "gridAccessMutex".


> So either it is being used as a shared lock -- to act as a "channel"
> whereby two or more objects can maintain some semantic
> interdependency. In such cases, I would submit that "sharedLock" is
> the minimum meaningful name. OTOH, you may have an object that only
> needs to protect some of its state. Offhand, I can't think of a
> convincing example, but say that an object needs to protect its
> output logging stream, and independently needs to protect the state
> of its interaction with some network server; in such cases it seems
> that the minimum meaningful names would be loggingLock and
> networkLock. The occurrence of the undecorated name "lock" is an
> indication that the author may not have understood what s/he was
> doing.

Sure, ok.


>
> (As, BTW, is casually moving a method between static and non-static
> -- that is a change at least as great, in OO terms, as moving a
> method between cl***es since it is changing the behaviour of two
> objects (treating the 'cl***' as if it were an object for these
> purposes).)

Of course, that's the point. But I see it /commonly/, and techniques for
newbies are best taught when they help limit such junior errors.


>
> -- chris

--
Whyowhydidn'tsunmakejavarequireanuppercaseletterto startcl***names....

Thomas G. Marshall wrote:
> Eric Sosman coughed up:
>
>>
>> In what way can the synchronized blocks (in this example
>>[snipped; see up-thread]
>>or in your follow-up with a static lock) "collide?" The code
>>is immutable whenever it's executable (that is, from the time
>>it's been loaded to the time when it's unloaded, if ever), so
>>it doesn't seem to be in need of much protection ...
>
> And how on earth do you know what threads are involved here?

I don't know and don't care; why should I need to? I'm
***uming there are N>1 threads because synchronization is not
required if N=1, but aside from that ...

> What if: The first method is called within one thread. The 2nd in another.
> Both of these access something that just cannot be accessed by more than
> one. This can involve something as innocuous as simple arithmetic on an
> integer primitive. Multiple threads doing very simple non-atomic things all
> at once can result in unpredictable results.

Agreed. But where does this integer primitive reside? If
two or more threads can all get at it, it must be in one of two
places: Either it's an instance variable of some Object and
you lock the Object to protect its state from getting mangled
or from being observed while inconsistent, or else it's a static
variable of some cl*** and you lock the cl*** object itself or
use a proxy. The integer primitive is part of a larger context.

>> The only kind of "collision" I can envision is if the two
>>pieces of code both manipulate some kind of shared resource.
>>Usually, that resource is a Java Object, and the Object is
>>the thing that needs the protection.
>
> It can be several objects, or simply an algorithm that cannot be
> interrupted. But it's not about saving the objects involved. It's about
> keeping multiple executions of segments of code blocked until one execution
> is done.

I disagree, vehemently. But I've already explained why and
my explanation didn't convince you, so there's not much use in
repeating it. We disagree, and there's an end on't.

> It is not the object in total that is protected. It is a section of code
> that you specifically force blockage into. If you instruct people to think
> that synchronization is protecting objects, then you are misleading them
> horribly. The bottom line is not that they are objects. The bottom line is
> that a section of executable code is protected from re-entrance by another
> thread.

... but I'm unable to resist temptation, because the last
sentence here is demonstrably false. Here we go:

cl*** Thing {
private int n = 0;
synchronized void increment() {
++n;
}
}

If you maintain that the code inside increment() can only be
executed by one thread at a time, you are flat-out wrong. It
can be executed simultaneously by ten threads, by a hundred,
by a thousand if you've got a big enough machine. All that's
required is that each of these threads operate on its own
instance of Thing; as long as no two of them try to manipulate
the same Thing at the same time, simultaneous execution is
completely unrestricted.

--
Eric.Sosman@sun.com

Eric Sosman coughed up:
> Thomas G. Marshall wrote:
>> Eric Sosman coughed up:
>>
>>>
>>> In what way can the synchronized blocks (in this example
> >>[snipped; see up-thread]
>>> or in your follow-up with a static lock) "collide?" The code
>>> is immutable whenever it's executable (that is, from the time
>>> it's been loaded to the time when it's unloaded, if ever), so
>>> it doesn't seem to be in need of much protection ...
>>
>> And how on earth do you know what threads are involved here?
>
> I don't know and don't care; why should I need to? I'm
> ***uming there are N>1 threads because synchronization is not
> required if N=1, but aside from that ...
>
>> What if: The first method is called within one thread. The 2nd in
>> another. Both of these access something that just cannot be accessed
>> by more than one. This can involve something as innocuous as simple
>> arithmetic on an integer primitive. Multiple threads doing very
>> simple non-atomic things all at once can result in unpredictable
>> results.
>
> Agreed. But where does this integer primitive reside? If
> two or more threads can all get at it, it must be in one of two
> places: Either it's an instance variable of some Object and
> you lock the Object to protect its state from getting mangled
> or from being observed while inconsistent, or else it's a static
> variable of some cl*** and you lock the cl*** object itself or
> use a proxy. The integer primitive is part of a larger context.

But there is much more to most objects than the /lines of code/ that are
protected from re-entrance. How far up in abstraction do you want to take
this context you refer to?

*Everything* has a larger context. So what? Let's see:

Lines within a synchronized{} protect the lines
if the lines are critical to an object, they protect the object
if the object is critical to a program, they protect the program
if the program is critical to national security, they protect
national security.

We're bouncing around the same thing.


>>> The only kind of "collision" I can envision is if the two
>>> pieces of code both manipulate some kind of shared resource.
>>> Usually, that resource is a Java Object, and the Object is
>>> the thing that needs the protection.
>>
>> It can be several objects, or simply an algorithm that cannot be
>> interrupted. But it's not about saving the objects involved. It's
>> about keeping multiple executions of segments of code blocked until
>> one execution is done.
>
> I disagree, vehemently. But I've already explained why and
> my explanation didn't convince you, so there's not much use in
> repeating it. We disagree, and there's an end on't.
>
>> It is not the object in total that is protected. It is a section of
>> code that you specifically force blockage into. If you instruct
>> people to think that synchronization is protecting objects, then you
>> are misleading them horribly. The bottom line is not that they are
>> objects. The bottom line is that a section of executable code is
>> protected from re-entrance by another thread.
>
> ... but I'm unable to resist temptation, because the last
> sentence here is demonstrably false. Here we go:
>
> cl*** Thing {
> private int n = 0;
> synchronized void increment() {
> ++n;
> }
> }

OF COURSE, and I've pointed that out already in this thread. All this shows
is that you can have multiple locks on a section of code. This (as I've
said) is why I prefer to teach newbies the following instead (of many
examples):

cl*** Thing
{
static Object incrementMutex = new Object();

private int n = 0;
void increment()
{
synchronized (incrementMutex)
{
++n;
}
}
}

...which is far more understandable once we add complexity. Further it stops
the static synchronization method mistake, and it allows us to further
refine the lines of code that are synchronized. The only drawback is that
if the entire method body appears within the block, it is [almost
immeasurably] slower.

....[snip]...


--
http://www.allexperts.com is a nifty way to get an answer to just about
/anything/.

Chris Uppal wrote:
>
> Thomas G. Marshall wrote:
>
> > It is not the object in total that is protected. It is a section of code
> > that you specifically force blockage into. If you instruct people to
> > think that synchronization is protecting objects, then you are misleading
> > them horribly. The bottom line is not that they are objects. The bottom
> > line is that a section of executable code is protected from re-entrance
> > by another thread.
>
> I disagree with this completely.
>
> First off, it is not -- no matter what your stance on OO, or even whether you
> are using an OO language -- code that is being protected, it is /state/. There
> are one or more variables whose values must be "managed" to allow correct
> concurrent access. So at minimum it is misleading to talk of protecting code,
> you should be talking about protecting data.

Agreed, except that you should say 'shared' state. Shared state is shared references and
cl*** and instance variables (but not local variables). Local variables can't be shared
with other threads.

Protecting code is the wrong concept. Perhaps it comes from the term, "critical
section". Basically, you only synchronize threads because they are accessing shared
state ... and you want to *protect data* against simultaneous access.

> Now in the OO world (and I'll accept, that Java is OO in this matter),
> data/state is normally bunched into semantically coherent units called objects.
> So the most natural thing to do is to manage concurrency at the level of the
> overall state of one object. In that case we are clearly using synchronisation
> to protect the (state of) the object. Just as Eric says. And the normal way
> to express that is with code synchronised on 'this' -- synchronised methods for
> instance.

Yes, saying that synchronization is for protecting objects is also misleading. It could
be part of an object -- it could be a loose confederation of several primitive variables
spread across multiple objects.

> Now in some cases, the state that needs to be protected will be either
> distributed across more than one object, or be less than the entire state of an
> object. Such cases are rare -- as you'd expect because the object is (or
> should be) expressing a semantically coherent unit of state, and the protection
> will normally be expected to follow the same boundaries, precisely /because/ it
> is protecting semantic coherence. Still, such cases do occur, and in those
> cases, and /only/ in those cases, your more "advanced" technique of using a
> lock object is appropriate.

Not quite. There are other cases where a separate 'monitor' is appropriate, but I agree
that it should be subject object in most cases.

> BTW, I don't think "lock" is a good name for a lock object. For the reasons
> given above, it is misleading to use it to protect exactly the state of 'this'.
>
> So either it is being used as a shared lock -- to act as a "channel" whereby
> two or more objects can maintain some semantic interdependency. In such cases,
> I would submit that "sharedLock" is the minimum meaningful name. OTOH, you may
> have an object that only needs to protect some of its state.

I generally use "monitor" as part of name for these.

--
Lee Fesperman, FFE Software, Inc. (http://www.firstsql.com)
================================================== ============
* The Ultimate DBMS is here!
* FirstSQL/J Object/Relational DBMS (http://www.firstsql.com)

Lee Fesperman coughed up:
> Chris Uppal wrote:
>>
>> Thomas G. Marshall wrote:
>>
>>> It is not the object in total that is protected. It is a section
>>> of code that you specifically force blockage into. If you instruct
>>> people to think that synchronization is protecting objects, then
>>> you are misleading them horribly. The bottom line is not that they
>>> are objects. The bottom line is that a section of executable code
>>> is protected from re-entrance by another thread.
>>
>> I disagree with this completely.
>>
>> First off, it is not -- no matter what your stance on OO, or even
>> whether you are using an OO language -- code that is being
>> protected, it is /state/. There are one or more variables whose
>> values must be "managed" to allow correct concurrent access. So at
>> minimum it is misleading to talk of protecting code, you should be
>> talking about protecting data.
>
> Agreed, except that you should say 'shared' state. Shared state is
> shared references and cl*** and instance variables (but not local
> variables). Local variables can't be shared with other threads.
>
> Protecting code is the wrong concept. Perhaps it comes from the term,
> "critical section". Basically, you only synchronize threads because
> they are accessing shared state ... and you want to *protect data*
> against simultaneous access.

Commonly, but no, not just data. For example, say you have a group of
things that you'd rather were executed all together without any one of them
being executed within another thread until you were done. You can make the
claim that there is data /somewhere/ being kept sane, but that "data" might
be just the total output to the user, or something equally vague, but that
is of no use here. For example, two threads executing these statements
(pseudo code):

1:
while (true)
{
for (i=0; i<10; i++)
{
System.out.print("-"); // Dashes
sleep(10 ms);
}
System.out.println();
}

2:
while (true)
{
for (i=0; i<10; i++)
{
System.out.print("X"); // X's
sleep(10 ms);
}
System.out.println();
}

To get these two threads to cooperate so that your output consists of
unbroken 10 character lines of dashes or x's, you need something akin to the
following. Basically add the synchronized() block to both threads
executions...

while (true)
{
// lineLock is java.lang.Object common instance between
// all threads executing this...
synchronized(lineLock)
{
for (i=0; i<10; i++)
{
System.out.print("-"); // Dashes
sleep(10 ms);
}
System.out.println();
}
}

So where's the data being protected? Just the output as data? See, it's
the lines of code within the synchronized block that are of issue here.
What you want to say that is protected as a result of it is up to you.

If anyone wants to point out that this in general doesn't strictly enforce
/alternating/ lines, we can discuss that later. You need a slightly
different mechanism for that, but this as is /does/ keep the lines whole.

IMHO the term "protect"{ing,ion} is what's leading us astray here.

In the following snippet:

synchronized(blaLock)
{
bla1;
bla2;
bla3;
}

The 3 lines within the synchronized block are "protected" from concurrent
access by another thread (using the same blaLock instance). And sure, given
this, protecting those 3 lines from re-entrance will protect the object or
whatever shared resource affected by the lines.

....[rip]...

--
Onedoctortoanother:"Ifthisismyrectalthermometer,wh erethehell'smypen???"

Thomas G. Marshall wrote:
>
> Lee Fesperman coughed up:
> >
> > Protecting code is the wrong concept. Perhaps it comes from the term,
> > "critical section". Basically, you only synchronize threads because
> > they are accessing shared state ... and you want to *protect data*
> > against simultaneous access.
>
> Commonly, but no, not just data. For example, say you have a group of
> things that you'd rather were executed all together without any one of them
> being executed within another thread until you were done. You can make the
> claim that there is data /somewhere/ being kept sane, but that "data" might
> be just the total output to the user, or something equally vague, but that
> is of no use here. For example, two threads executing these statements
> (pseudo code):
>
> 1:
> while (true)
> {
> for (i=0; i<10; i++)
> {
> System.out.print("-"); // Dashes
> sleep(10 ms);
> }
> System.out.println();
> }
>
> 2:
> while (true)
> {
> for (i=0; i<10; i++)
> {
> System.out.print("X"); // X's
> sleep(10 ms);
> }
> System.out.println();
> }
>
> To get these two threads to cooperate so that your output consists of
> unbroken 10 character lines of dashes or x's, you need something akin to the
> following. Basically add the synchronized() block to both threads
> executions...
>
> while (true)
> {
> // lineLock is java.lang.Object common instance between
> // all threads executing this...
> synchronized(lineLock)
> {
> for (i=0; i<10; i++)
> {
> System.out.print("-"); // Dashes
> sleep(10 ms);
> }
> System.out.println();
> }
> }
>
> So where's the data being protected? Just the output as data? See, it's
> the lines of code within the synchronized block that are of issue here.
> What you want to say that is protected as a result of it is up to you.

The data being protected is the System.out object. It is a wrapper around an external
resource which certainly has state. It also might have a buffer that needs protection.
This is abstracted at the object (PrintStream) level, and thus synchronized at that
level.

> If anyone wants to point out that this in general doesn't strictly enforce
> /alternating/ lines, we can discuss that later. You need a slightly
> different mechanism for that, but this as is /does/ keep the lines whole.
>
> IMHO the term "protect"{ing,ion} is what's leading us astray here.
>
> In the following snippet:
>
> synchronized(blaLock)
> {
> bla1;
> bla2;
> bla3;
> }
>
> The 3 lines within the synchronized block are "protected" from concurrent
> access by another thread (using the same blaLock instance). And sure, given
> this, protecting those 3 lines from re-entrance will protect the object or
> whatever shared resource affected by the lines.

Simply put: if you are not protecting data then synchronization is meaningless. It's not
doing anything. There is no useful case where this is not true.

--
Lee Fesperman, FFE Software, Inc. (http://www.firstsql.com)
================================================== ============
* The Ultimate DBMS is here!
* FirstSQL/J Object/Relational DBMS (http://www.firstsql.com)

Lee Fesperman wrote:

> > Now in some cases, the state that needs to be protected will be either
> > distributed across more than one object, or be less than the entire
> > state of an object. Such cases are rare -- as you'd expect because the
> > object is (or
> > should be) expressing a semantically coherent unit of state, and the
> > protection
> > will normally be expected to follow the same boundaries, precisely
> > /because/ it
> > is protecting semantic coherence. Still, such cases do occur, and in
> > those
> > cases, and /only/ in those cases, your more "advanced" technique of
> > using a
> > lock object is appropriate.
>
> Not quite. There are other cases where a separate 'monitor' is
> appropriate, but I agree that it should be subject object in most cases.

Well, the data to be protected is either a subset of the state of one object,
the entire state of one object, or spread across more than one object -- it's
not immediately obvious what your fourth possibility is ?

-- chris

Thomas G. Marshall wrote:
> Steve Horsley coughed up:

I hope that's an automated insult and not a personal one.

>
> For similar reasons I *strongly* recommend that a newbie not use
> synchronized methods such as:
>
> public synchronized void hoohaa()
> {
> //...goblidy gook...
> }
>
> but instead use
>
> public void hoohaa()
> {
> synchronized (myLock)
> {
> //...goblidy gook...
> }
> }
>
> and have myLock be an Object instance devoted to nothing but holding the
> lock.
>
> The reason for this is important:
>
> a. it keeps the confusion down in general about whether or not a lock is
> held by the same instance as used elsewhere.
>
> b. it keeps the following bug out. This bug I've seen time and time again:
> someone tries to add a synchronized method, but later makes it static for
> some other reason. But he forgets that synchronized then is locking on the
> cl*** object when used with static. Two locks in that case.

Yes, but your approach does remove the possibility of externally syncing on
an object while you call a sequence of its methods, in the way you might
lock and then iterate a List or array. Or worse, a user might _***ume_
that he is succesfully keeping other threads out when he isn't. Like this:

synchronised(myThingy) {
myThingy.doThis();
myThingy.doThat();
myThingy.doTheOther();
}

Maybe myThingy should expose the lock object? :

public final Object syncLock = new Object();

To be lazy, you could use yourself as the syncLock:

public final Object syncLock = this;

but then you're pretty-much back to standard synchronized methods.
Maybe the idiom ought to be:

public void doThis() {
synchronized(this) {
...
}
}

Making the above static will not compile.
And it seems more intuitive than just making the method synchronized.

Swings and roundabouts, horses for courses. Just thinking aloud.

Steve.


There are

Steve Horsley coughed up:
> Thomas G. Marshall wrote:
>> Steve Horsley coughed up:
>
> I hope that's an automated insult and not a personal one.

Of course. No insult, please just see it as quirky.


>> For similar reasons I *strongly* recommend that a newbie not use
>> synchronized methods such as:
>>
>> public synchronized void hoohaa()
>> {
>> //...goblidy gook...
>> }
>>
>> but instead use
>>
>> public void hoohaa()
>> {
>> synchronized (myLock)
>> {
>> //...goblidy gook...
>> }
>> }
>>
>> and have myLock be an Object instance devoted to nothing but holding
>> the lock.
>>
>> The reason for this is important:
>>
>> a. it keeps the confusion down in general about whether or not a
>> lock is held by the same instance as used elsewhere.
>>
>> b. it keeps the following bug out. This bug I've seen time and time
>> again: someone tries to add a synchronized method, but later makes
>> it static for some other reason. But he forgets that synchronized
>> then is locking on the cl*** object when used with static. Two
>> locks in that case.
>
> Yes, but your approach does remove the possibility of externally
> syncing on an object while you call a sequence of its methods, in the
> way you might lock and then iterate a List or array.

Yep, that's day 2 in the cl***. lol. However, remember that the caveat is
that having an object use itself as the holder of the lock, IME has lead to
"magical thinking" by junior engineers. "Externally, all you need to do is
synchronize on the object and it all works", and words to that effect are
fairly prevalent.


> Or worse, a user
> might _***ume_ that he is succesfully keeping other threads out when
> he isn't. Like this:
>
> synchronised(myThingy) {
> myThingy.doThis();
> myThingy.doThat();
> myThingy.doTheOther();
> }

Well that's what makes the technique so dangerous in the first place.
Cl***es that allow their objects to be synchronized need to be very tightly
do***ented.


> Maybe myThingy should expose the lock object? :

Most of the time this isn't necessary, and it's important to start newcomers
off with explicitly identifying the object holding the lock, for the reasons
I've explained.

But using the object itself as its own lock is certainly ok. So long as the
correct idiom is employed (see below).


> public final Object syncLock = new Object();
>
> To be lazy, you could use yourself as the syncLock:
>
> public final Object syncLock = this;

This actually is not as lazy, nor as gross, as you might think.


> but then you're pretty-much back to standard synchronized methods.
> Maybe the idiom ought to be:
>
> public void doThis() {
> synchronized(this) {
> ...
> }
> }
>
> Making the above static will not compile.
> And it seems more intuitive than just making the method synchronized.

YES. But I stress the /synchronized(this)/ idiom because interesting things
happen when the /synchronized method()/ idiom is used:

1. The developer is far less likely to shift over to
using a different lock should the need arise. He
feels "locked in" to using the current object
instance.

2. The developer is far less likely to start paring
down the critical sections of his methods, should
things get mindblowingly slow.

#2 of course allows this:

public void method()
{
line 1;
synchronized(this)
{
line 2;
line 3;
}
line 4;
}

>
> Swings and roundabouts, horses for courses. Just thinking aloud.

Sure!


> Steve.
>
>
> There are

.....aliens in area 51?

--
It'salwaysbeenmygoalinlifetocreateasignaturethaten dedwiththeword"blarphoogy"
..

Lee Fesperman coughed up:
> Thomas G. Marshall wrote:


....[rip]...


>> IMHO the term "protect"{ing,ion} is what's leading us astray here.
>>
>> In the following snippet:
>>
>> synchronized(blaLock)
>> {
>> bla1;
>> bla2;
>> bla3;
>> }
>>
>> The 3 lines within the synchronized block are "protected" from
>> concurrent access by another thread (using the same blaLock
>> instance). And sure, given this, protecting those 3 lines from
>> re-entrance will protect the object or whatever shared resource
>> affected by the lines.
>
> Simply put: if you are not protecting data then synchronization is
> meaningless. It's not doing anything. There is no useful case where
> this is not true.

Ok. And if you are not protecting the program at large, then
synchronization is meaningless. There is no useful case where this is not
true. Keep wandering up, like I pointed out elsethread.

Larger contexts always exist. But there are /lines of code/ in
synchronization blocks and methods. Saying that you are protecting an
object is of no good in teaching someone how the particular /lines of code/
might need be atomic to an algorithm.

And it is those lines of code /within/ such synchronization structures that
are protected. Lines exterior to those simply are not.


--
It'salwaysbeenmygoalinlifetocreateasignaturethaten dedwiththeword"blarphoogy"
..