[std-proposals] class black and white, separate interface from content

[mobiphil]

Hi there,

maybe somebody can join and help with this following brainstorming, may
sound naive, but would love to hear opinions.

Two new keywords: (white and black), or (iface and impl) or anything else,
I will go for the moment with two rather neutral keywords black and white.
Black is for blackbox, white is for whitebox. Black is for what you hide,
white is what you show. Black is really private, white is really public. No
more pimpl, guaranteed fast compilation times, less recompilation and the
list does not stop here.

// content of A.h
class white A {
/* here come all public virtuals and non virtuals */
/* we could also allow public data members */
};


// content of A.cpp
class black A {
/* here come all data members */
/* here come all the private methods */
};

/* how would I solve */
A * = new A();

what do we need for allocation? The size of the data members from white and
black? Compiler can either generate a classsize() function or with -fPIC
trick, put the size into data section, so that it can be available at
runtime. Advantage of a classize() function is that like this, classes can
have an easier life in shared libraries (plugin pattern for instance).
What about allocating on the stack? Do the same in alloca style.

What about subclassing? Well here are two problems: the size and the
offsets.

Subclass size: For the subclass the classize will be it's size plus the
data it defines additionally.

Offsets: we either force all the public data members into the blackbox (so
only private data members, thus defined in black), and outside world does
not have to know about offsets, everything will be accessed only through
getters/setters. Or one can use the normal offset calculation like for
normal classes, one can make the convention that "white" members are at the
beginning of the object. The problem would be a bit tricky with subclasses.
But also here the compiler can calculate offests as: offset in subclass +
size of baseclass.

thanks for your feedback

[Douglas Boffey]

While the concept sounds good on the surface, there seems an awful lot
of baggage associated with it.

Post by mobiphil
Hi there,
maybe somebody can join and help with this following brainstorming, may
sound naive, but would love to hear opinions.
Two new keywords: (white and black), or (iface and impl) or anything else,
I will go for the moment with two rather neutral keywords black and white.
Black is for blackbox, white is for whitebox. Black is for what you hide,
white is what you show. Black is really private, white is really public. No
more pimpl, guaranteed fast compilation times, less recompilation and the
list does not stop here.
// content of A.h
class white A {
/* here come all public virtuals and non virtuals */
/* we could also allow public data members */
};
// content of A.cpp
class black A {
/* here come all data members */
/* here come all the private methods */
};
/* how would I solve */
A * = new A();
what do we need for allocation? The size of the data members from white and
black? Compiler can either generate a classsize() function or with -fPIC
trick, put the size into data section, so that it can be available at
runtime. Advantage of a classize() function is that like this, classes can
have an easier life in shared libraries (plugin pattern for instance).
What about allocating on the stack? Do the same in alloca style.
What about subclassing? Well here are two problems: the size and the
offsets.
Subclass size: For the subclass the classize will be it's size plus the
data it defines additionally.
Offsets: we either force all the public data members into the blackbox (so
only private data members, thus defined in black), and outside world does
not have to know about offsets, everything will be accessed only through
getters/setters. Or one can use the normal offset calculation like for
normal classes, one can make the convention that "white" members are at the
beginning of the object. The problem would be a bit tricky with subclasses.
But also here the compiler can calculate offests as: offset in subclass +
size of baseclass.
thanks for your feedback
--
---
You received this message because you are subscribed to the Google Groups
"ISO C++ Standard - Future Proposals" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at
http://groups.google.com/a/isocpp.org/group/std-proposals/.

[Douglas Boffey]

Have you had a look at the work done by the Modules SG?

Post by Douglas Boffey
While the concept sounds good on the surface, there seems an awful lot
of baggage associated with it.

Post by mobiphil
Hi there,
maybe somebody can join and help with this following brainstorming, may
sound naive, but would love to hear opinions.
Two new keywords: (white and black), or (iface and impl) or anything else,
I will go for the moment with two rather neutral keywords black and white.
Black is for blackbox, white is for whitebox. Black is for what you hide,
white is what you show. Black is really private, white is really public. No
more pimpl, guaranteed fast compilation times, less recompilation and the
list does not stop here.
// content of A.h
class white A {
/* here come all public virtuals and non virtuals */
/* we could also allow public data members */
};
// content of A.cpp
class black A {
/* here come all data members */
/* here come all the private methods */
};
/* how would I solve */
A * = new A();
what do we need for allocation? The size of the data members from white and
black? Compiler can either generate a classsize() function or with -fPIC
trick, put the size into data section, so that it can be available at
runtime. Advantage of a classize() function is that like this, classes can
have an easier life in shared libraries (plugin pattern for instance).
What about allocating on the stack? Do the same in alloca style.
What about subclassing? Well here are two problems: the size and the
offsets.
Subclass size: For the subclass the classize will be it's size plus the
data it defines additionally.
Offsets: we either force all the public data members into the blackbox (so
only private data members, thus defined in black), and outside world does
not have to know about offsets, everything will be accessed only through
getters/setters. Or one can use the normal offset calculation like for
normal classes, one can make the convention that "white" members are at the
beginning of the object. The problem would be a bit tricky with subclasses.
But also here the compiler can calculate offests as: offset in subclass +
size of baseclass.
thanks for your feedback
--
---
You received this message because you are subscribed to the Google Groups
"ISO C++ Standard - Future Proposals" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at
http://groups.google.com/a/isocpp.org/group/std-proposals/.

[mobiphil]

Post by Douglas Boffey
Have you had a look at the work done by the Modules SG?

Yes, but they are almost orthogonally different, though their effect may
overlap. I am hacking right now the modules implementation of clang.

[mobiphil]

Post by Douglas Boffey
While the concept sounds good on the surface, there seems an awful lot
of baggage associated with it.
I know... but that is the objective of this proposed brainstorming, to

name all the elements... What I forgot to mention is that obviously using
the keywords thus splitting the class would be obviously optional

[Jean-Marc Bourguet]

If you go that road, I suggest looking at Modula 3 and its "partial
revelations", ie the possibility for a component to provides different
interfaces to different customers.

Yours,
--
Jean-Marc

[mobiphil]

Post by Jean-Marc Bourguet
If you go that road, I suggest looking at Modula 3 and its "partial
revelations", ie the possibility for a component to provides different
interfaces to different customers.

was not aware about this, skimmed the article (link below), will read it
more carefully at a later time, but it seems that it is a generalization of
what I just proposed.

What is ill in C++ is that one tried to push everything into one class
declaration. I was thinking also about including in the same proposal the
fact that non virtual functions declarations could be easily removed from
the class declaration and allow anywhere just Class::method() {}.. kind of
declarations/definitions. This would allow moving of different details of a
class into different header. Obviously This does not mean that one wants to
create a class that can do everything.

The "black" I proposed, could be split into several blacks, the class
black idiom could be regarded as even superfluous, and just allow "int
Class::member ;" declaration anywhere. The linker could gather all the
parts of the classes and reason about it. But this later detail is a bit
extreme, I agree.



http://www.cs.tut.fi/lintula/manual/modula3/modula-3/html/partial-rev/index.html

[Tom Honermann]

Post by mobiphil
maybe somebody can join and help with this following brainstorming, may
sound naive, but would love to hear opinions.

Are you familiar with Objective-C? If not, I recommend researching
Objective-C interfaces, implementations, categories, and class
extensions to help flesh out your idea.

What would sizeof(A) produce in a translation unit that observes only
the interface (white) declaration? Would this require a dynamic sizeof
similar to that required for C99 variable length arrays? (I think the
need for a dynamic sizeof is one of the issues that have prevented
variable length arrays being added to C++).

Would class data members of these split class types be allowed when only
the interface (white) declaration is present in a translation unit? If
so, what would be the result of sizeof(X) for such a class?

Tom.

[mobiphil]

Thanks for joining

Post by mobiphil
maybe somebody can join and help with this following brainstorming, may

Post by mobiphil
sound naive, but would love to hear opinions.

Are you familiar with Objective-C? If not, I recommend researching
Objective-C interfaces, implementations, categories, and class
extensions to help flesh out your idea.

Have some very basic notion. I would not spend too much time for further
study if there is zero acceptance for such ideas.

Post by mobiphil
What would sizeof(A) produce in a translation unit that observes only
the interface (white) declaration?

Would this require a dynamic sizeof

Post by mobiphil
similar to that required for C99 variable length arrays? (I think the
need for a dynamic sizeof is one of the issues that have prevented
variable length arrays being added to C++).
Compilation error for static sizeof. If really needed for runtime, then

replace with a pointer to some global text that would be solved by the
linker (like fpic). Or go for the function Class::sizeof() that would be
automatically generated by the compiler.

Post by mobiphil
Would class data members of these split class types be allowed when only
the interface (white) declaration is present in a translation unit? If
so, what would be the result of sizeof(X) for such a class?

static sizeof(x), does not make sense, we both agree. I am not supposed to
want know the color, shape etc. of an object that does not exist.
If you do not provide the blackbox, you will have linker error, the same
way you would have for missing vtables. This linker error would be
"implemented" either by missing Class::size() function or Class::size
global variable.

[Thiago Macieira]

Post by mobiphil
static sizeof(x), does not make sense, we both agree. I am not supposed to
want know the color, shape etc. of an object that does not exist.
If you do not provide the blackbox, you will have linker error, the same
way you would have for missing vtables. This linker error would be
"implemented" either by missing Class::size() function or Class::size
global variable.

Without a static sizeof, you can't:

- use operator new
- use make_shared or make_unique
- declare the object on the stack
- use it as a member of another object

The only way you can create such an object is to call an allocator function.
--
Thiago Macieira - thiago (AT) macieira.info - thiago (AT) kde.org
Software Architect - Intel Open Source Technology Center
PGP/GPG: 0x6EF45358; fingerprint:
E067 918B B660 DBD1 105C 966C 33F5 F005 6EF4 5358

[mobiphil]

I thought I touch all the subjects, but I will comment once more

Post by Thiago Macieira
- use operator new

Not if you do not separate the concepts. When does new happen? At runtime.
Do I know the size of the object at runtime yes. Proposed here either a
function that the compiler would generate, or a global that would hold the
size of the object deduced based on the size of "black" (+ size of white if
I want to allow data in the white)

Post by Thiago Macieira
- use make_shared or make_unique

Well, did not look into this one, but not all objects or classes fit into
all patterns. Non copiables do not fit into patterns that need copiable
etc...

Post by Thiago Macieira
- declare the object on the stack

Yes you could, as mentioned earlier, do alloca based on the dynamic size
mentioned above

Post by Thiago Macieira
- use it as a member of another object

Yes you could. The size and offset needed though would be calculated at
runtime with similare mechasism like -fpic

[Thiago Macieira]

Post by mobiphil

Post by Thiago Macieira
- use it as a member of another object

Yes you could. The size and offset needed though would be calculated at
runtime with similare mechasism like -fpic

More like virtual inheritance instead of position-independent code.
--
Thiago Macieira - thiago (AT) macieira.info - thiago (AT) kde.org
Software Architect - Intel Open Source Technology Center
PGP/GPG: 0x6EF45358; fingerprint:
E067 918B B660 DBD1 105C 966C 33F5 F005 6EF4 5358

[mobiphil]

Post by Thiago Macieira

Post by mobiphil
Yes you could. The size and offset needed though would be calculated at
runtime with similare mechasism like -fpic

More like virtual inheritance instead of position-independent code.

What do you mean?

What I mean with "similar mechanism like -fPIC": offsets and size will be
known at runtime by pointing to a global location that will be resolved by
the linker

[Thiago Macieira]

Post by mobiphil

Post by Thiago Macieira

Post by mobiphil
Yes you could. The size and offset needed though would be calculated at
runtime with similare mechasism like -fpic

More like virtual inheritance instead of position-independent code.

What do you mean?
What I mean with "similar mechanism like -fPIC": offsets and size will be
known at runtime by pointing to a global location that will be resolved by
the linker

Also possible, but just not the way I envisioned it. Loading it from a very
different page of memory may be slower than keeping it local and just
referencing each dynamic item by a pointer, like a virtual base.
--
Thiago Macieira - thiago (AT) macieira.info - thiago (AT) kde.org
Software Architect - Intel Open Source Technology Center
PGP/GPG: 0x6EF45358; fingerprint:
E067 918B B660 DBD1 105C 966C 33F5 F005 6EF4 5358

[mobiphil]

Post by Thiago Macieira
Also possible, but just not the way I envisioned it. Loading it from a very
different page of memory may be slower than keeping it local and just
referencing each dynamic item by a pointer, like a virtual base.

I had this in mind. I am trying to think both in terms of dynamic and
static linking. Dynamic linking is probably bringing more complex, I was
rather thinking in terms of it. Without going into details dynamic linking
with current compilers/linkers can be mainly done in two ways: either
intrusive or non intrusive. The linker may go to the code and patch
function addresses, or use the function wrappers. (sorry if I do not use
the right terminology). All these are implementation details that have
their advantages/disadvantages. As most of shared libraries are implemented
with -fpic, for the sake of simplicity I mentioned this one. I am perfectly
aware of the expense. Patching references at dynamic linking time would be
another solution. Patching all the references has the disadvantage that you
may patch 99% of references that are not used, slowing down thus dynamic
linking. If it is static linking the linker could patch it at linking time.

Referencing each dynamic item by a pointer : you are proposing that dynamic
items shall be allocated separately and keep a pointer like in the case of
opaque pointers? Do not think this is a solution. These pointers equally
can fall outside of the page and have same cache issues you are touched or?
For me it is important that the object itself stays compact. The question
is what is the best solution to access members of the "white". The most
straightforward would be for the sake of simplicity to forbid data members
in the white, and really keep it like an interface.

[Thiago Macieira]

Post by mobiphil
Referencing each dynamic item by a pointer : you are proposing that dynamic
items shall be allocated separately and keep a pointer like in the case of
opaque pointers?

No, I am not proposing that. First of all, I am not proposing anything. I
personally don't think your proposal will fly...

But if it were to be implemented, I'd expect it to be done like a class with
virtual bases: the allocator allocates a block of memory with sufficient size
for all of the dynamic objects inside, which it calculates at runtime by
adding up the sizes of each one, and initialises a helper pointer to those
objects. This extra pointer is an implementation detail and is hidden from the
user, just like a virtual base's pointer.

The other solution is to create a "dynamic table" of the class, which like the
virtual table, is associated one per class. This table would contain the size
of the full class and the offsets of each of the dynamic-sized members. The
problem here is initialising this table, since it can only be done at runtime,
it has to be done on program load.
--
Thiago Macieira - thiago (AT) macieira.info - thiago (AT) kde.org
Software Architect - Intel Open Source Technology Center
PGP/GPG: 0x6EF45358; fingerprint:
E067 918B B660 DBD1 105C 966C 33F5 F005 6EF4 5358

[Douglas Boffey]

The other option would be for the compiler to produce an interface
file containing the necessary details.

Post by Thiago Macieira

Post by mobiphil
Referencing each dynamic item by a pointer : you are proposing that dynamic
items shall be allocated separately and keep a pointer like in the case of
opaque pointers?

No, I am not proposing that. First of all, I am not proposing anything. I
personally don't think your proposal will fly...
But if it were to be implemented, I'd expect it to be done like a class with
virtual bases: the allocator allocates a block of memory with sufficient size
for all of the dynamic objects inside, which it calculates at runtime by
adding up the sizes of each one, and initialises a helper pointer to those
objects. This extra pointer is an implementation detail and is hidden from the
user, just like a virtual base's pointer.
The other solution is to create a "dynamic table" of the class, which like the
virtual table, is associated one per class. This table would contain the size
of the full class and the offsets of each of the dynamic-sized members. The
problem here is initialising this table, since it can only be done at runtime,
it has to be done on program load.
--
Thiago Macieira - thiago (AT) macieira.info - thiago (AT) kde.org
Software Architect - Intel Open Source Technology Center
E067 918B B660 DBD1 105C 966C 33F5 F005 6EF4 5358
--
---
You received this message because you are subscribed to the Google Groups
"ISO C++ Standard - Future Proposals" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at
http://groups.google.com/a/isocpp.org/group/std-proposals/.

[mobiphil]

Post by Douglas Boffey
The other option would be for the compiler to produce an interface
file containing the necessary details.

I am afraid that would still introduce compile time dependencies. Once you
changed the content of the black, that would regenerate the interface thus
force the dependees to recompile.

[Douglas Boffey]

... and references that when compiling the dependent TU.

Post by Thiago Macieira

Post by mobiphil
Referencing each dynamic item by a pointer : you are proposing that dynamic
items shall be allocated separately and keep a pointer like in the case of
opaque pointers?

No, I am not proposing that. First of all, I am not proposing anything. I
personally don't think your proposal will fly...
But if it were to be implemented, I'd expect it to be done like a class with
virtual bases: the allocator allocates a block of memory with sufficient size
for all of the dynamic objects inside, which it calculates at runtime by
adding up the sizes of each one, and initialises a helper pointer to those
objects. This extra pointer is an implementation detail and is hidden from the
user, just like a virtual base's pointer.
The other solution is to create a "dynamic table" of the class, which like the
virtual table, is associated one per class. This table would contain the size
of the full class and the offsets of each of the dynamic-sized members. The
problem here is initialising this table, since it can only be done at runtime,
it has to be done on program load.
--
Thiago Macieira - thiago (AT) macieira.info - thiago (AT) kde.org
Software Architect - Intel Open Source Technology Center
E067 918B B660 DBD1 105C 966C 33F5 F005 6EF4 5358
--
---
You received this message because you are subscribed to the Google Groups
"ISO C++ Standard - Future Proposals" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at
http://groups.google.com/a/isocpp.org/group/std-proposals/.

[mobiphil]

Post by Tom Honermann

Post by mobiphil
Referencing each dynamic item by a pointer : you are proposing that

dynamic

Post by mobiphil
items shall be allocated separately and keep a pointer like in the case

of

Post by mobiphil
opaque pointers?

No, I am not proposing that. First of all, I am not proposing anything.

Well, suggesting, writing, proposing, I think you get it...

Post by Tom Honermann
personally don't think your proposal will fly...

That is a bit negative. Seeing how slowly C++ developed, and how rigid it
is, that would not surprise me. Unfortunately most of industries failed to
influence the evolution of C++. Such a feature would have saved for most of
projects I worked lot of dollars. I see in your signature that you work in
kde area, thus probably you deal lot with QT. I wonder if having such
feature QT would have bothered with opaque pointers.

Post by Tom Honermann
But if it were to be implemented, I'd expect it to be done like a class with
virtual bases: the allocator allocates a block of memory with sufficient size
for all of the dynamic objects inside,

sufficient you mean allocated can exceed much the size of all dynamic
objects?

Post by Tom Honermann
which it calculates at runtime by
adding up the sizes of each one, and initialises a helper pointer to those
objects.

Sorry: one pointer to several objects, is that correct? Who would store
this pointer, how would it work?

Post by Tom Honermann
The other solution is to create a "dynamic table" of the class, which like the
virtual table, is associated one per class. This table would contain the size
of the full class and the offsets of each of the dynamic-sized members. The
problem here is initialising this table, since it can only be done at runtime,
it has to be done on program load.

You get it... This is exactly what I did mean with the -fpic like
mechanism, was bit lazy to describe.
But you made a good point and will link it with your previous observation
about having this information in a different memory page that will probably
cause cache miss. Well unfortunately virtual tables suffer of the same
problem. So we arrived to the point: this information could be part of the
virtual pointer table.

Another solution would be (at least for the beta :) ) to forbid data
members in the interface. The dynamic linker or the linker (I know, linkers
do not have such tasks) could go only through the code that knows about the
black, and patch it with right offset, as he would already know about the
blacks of superclases.

[Thiago Macieira]

Post by mobiphil

Post by Thiago Macieira
But if it were to be implemented, I'd expect it to be done like a class with
virtual bases: the allocator allocates a block of memory with sufficient size
for all of the dynamic objects inside,

sufficient you mean allocated can exceed much the size of all dynamic
objects?

Exceeds yes. By the padding amount and the extra pointers.

Post by mobiphil

Post by Thiago Macieira
which it calculates at runtime by
adding up the sizes of each one, and initialises a helper pointer to those
objects.

Sorry: one pointer to several objects, is that correct? Who would store
this pointer, how would it work?

Let's say Foo and Bar are dynamic in the object below:

class MyObject : public virtual SomeOther
{
Foo foo;
int i;
int j;
void *d;
Bar bar;
virtual ~MyObject();
};

If it's implemented as I described, the real object's ABI is:

struct MyObject_real
{
MyObject_vtable *_vptr;
SomeOther *_vbase;
Foo *foo;
Bar *bar;

int i;
int j;
void *d;

SomeOther _vbase_space;
// plus space for Foo and Bar
};

The code that allocates MyObject needs to take sizeof(MyObject_real) and add
the dynamic sizes of Foo and Bar. The difference between that and the allocated
space for the virtual base is that the size of the former isn't known at
compile time.

If you do object.foo.x, it's implemented as real(object).foo->x.

Post by mobiphil

Post by Thiago Macieira
The other solution is to create a "dynamic table" of the class, which like the
virtual table, is associated one per class. This table would contain the size
of the full class and the offsets of each of the dynamic-sized members. The
problem here is initialising this table, since it can only be done at runtime,
it has to be done on program load.

You get it... This is exactly what I did mean with the -fpic like
mechanism, was bit lazy to describe.
But you made a good point and will link it with your previous observation
about having this information in a different memory page that will probably
cause cache miss. Well unfortunately virtual tables suffer of the same
problem. So we arrived to the point: this information could be part of the
virtual pointer table.

This requires careful research by the ABI developers to see which solution is
more efficient. My educated guess is that mine above is more cache- and
instruction-efficient, but I have not (and have no intention to) actually
benchmarked this.

Your solution requires loading a ptrdiff_t from a known, global location in
memory, then adding it to your class's base pointer. With -fPIC, this becomes
a double indirection: you need to load a pointer from the GOT and then load
the ptrdiff_t from the address the GOT contained, only then add it.

And then there's the initialisation of this table. For a single dynamic-sized
object, the value can be emitted by the compiler at compile time. For an
object that contains other dynamic-sized objects, the compiler can't do it, so
it has to emit load-time code to calculate this and save in the "dynamic
table". I don't want to imagine the problems caused by getting the
initialisation order wrong.
--
Thiago Macieira - thiago (AT) macieira.info - thiago (AT) kde.org
Software Architect - Intel Open Source Technology Center
PGP/GPG: 0x6EF45358; fingerprint:
E067 918B B660 DBD1 105C 966C 33F5 F005 6EF4 5358

[mobiphil]

Post by Thiago Macieira
struct MyObject_real

{

Post by Thiago Macieira
MyObject_vtable *_vptr;
SomeOther *_vbase;
Foo *foo;
Bar *bar;
int i;
int j;
void *d;
SomeOther _vbase_space;
// plus space for Foo and Bar
};

I see, but personally I would not be happy seeing the space wasted for
pointers especially not 64 bit. I am thinking to make another proposal
about 64bit pointer compression. Eventually offsets would do it. Why should
I store the layout of each object in each object. And that is what I mean
by "this information could be part of the virtual pointer table". It is
true that vtable is not present for all classes, but well, there will be
another reason for the existence of vtable.

Post by Thiago Macieira
The code that allocates MyObject needs to take sizeof(MyObject_real) and add
the dynamic sizes of Foo and Bar. The difference between that and the allocated
space for the virtual base is that the size of the former isn't known at
compile time.

Regarding layout I think the base should come first. Isn't it implemented
like that. Casting normally keeps the address, so base class data should be
at lower addresses.

Well, I think better not to mix -fpic anymore and mix dynamic and static
linking for the moment. When I wrote the first message, did not think about
virtual tables. Probably the cleanest is to say size and offset (layout)
information will be available at runtime in vtable or vtable alike
structure.

[Thiago Macieira]

Post by mobiphil
Regarding layout I think the base should come first. Isn't it implemented
like that. Casting normally keeps the address, so base class data should be
at lower addresses.

Not for virtual bases. The virtual base is kept after the non-virtuals.
--
Thiago Macieira - thiago (AT) macieira.info - thiago (AT) kde.org
Software Architect - Intel Open Source Technology Center
PGP/GPG: 0x6EF45358; fingerprint:
E067 918B B660 DBD1 105C 966C 33F5 F005 6EF4 5358

[mobi phil]

so, this topic can be forgotten, nobody bites? ;)

[Thiago Macieira]

Post by mobi phil
so, this topic can be forgotten, nobody bites? ;)

You haven't written a proposal.
--
Thiago Macieira - thiago (AT) macieira.info - thiago (AT) kde.org
Software Architect - Intel Open Source Technology Center
PGP/GPG: 0x6EF45358; fingerprint:
E067 918B B660 DBD1 105C 966C 33F5 F005 6EF4 5358

[mobi phil]

Post by Thiago Macieira

Post by mobi phil
so, this topic can be forgotten, nobody bites? ;)

You haven't written a proposal.

Sorry, do not understand this. How would you pretend there was no proposal
while you reacted to it yourself....

[Thiago Macieira]

Post by mobi phil

Post by Thiago Macieira

Post by mobi phil
so, this topic can be forgotten, nobody bites? ;)

You haven't written a proposal.

Sorry, do not understand this. How would you pretend there was no proposal
while you reacted to it yourself....

I meant that you did not write a paper and submit to the committee for
consideration.

I think your idea has merit. It makes sense to present it and talk to the
people doing Modules. Some aspects of it could be adopted there.
--
Thiago Macieira - thiago (AT) macieira.info - thiago (AT) kde.org
Software Architect - Intel Open Source Technology Center
PGP/GPG: 0x6EF45358; fingerprint:
E067 918B B660 DBD1 105C 966C 33F5 F005 6EF4 5358

[mobi phil]

Post by mobiphil

Post by mobi phil

Post by Thiago Macieira

Post by mobi phil
so, this topic can be forgotten, nobody bites? ;)

You haven't written a proposal.

Sorry, do not understand this. How would you pretend there was no

proposal

Post by mobi phil
while you reacted to it yourself....

I meant that you did not write a paper and submit to the committee for
consideration.
I think your idea has merit. It makes sense to present it and talk to the
people doing Modules. Some aspects of it could be adopted there.

While I do not know exactly the formal procedure, I do not think it is
worth the effort to write a formal paper unless some of the honorable
members of the honorable committee would show minimum interest to ever
consider it. The reason of the brainstorming is to find together weaknesses
of the model and solutions for them.

[g***@gmail.com]

Post by mobi phil

<javascript:>>

Post by mobi phil

Post by Thiago Macieira

Post by mobi phil
so, this topic can be forgotten, nobody bites? ;)

You haven't written a proposal.

Sorry, do not understand this. How would you pretend there was no

proposal

Post by mobi phil
while you reacted to it yourself....

I meant that you did not write a paper and submit to the committee for
consideration.
I think your idea has merit. It makes sense to present it and talk to the
people doing Modules. Some aspects of it could be adopted there.

While I do not know exactly the formal procedure, I do not think it is
worth the effort to write a formal paper unless some of the honorable
members of the honorable committee would show minimum interest to ever
consider it. The reason of the brainstorming is to find together weaknesses
of the model and solutions for them.
I agree

[Ville Voutilainen]

Post by mobiphil

Post by mobi phil
While I do not know exactly the formal procedure, I do not think it is
worth the effort to write a formal paper unless some of the honorable
members of the honorable committee would show minimum interest to ever
consider it. The reason of the brainstorming is to find together weaknesses
of the model and solutions for them.

I agree

As far as I understand it, the idea deals with being able to express
interface-implementation splits in a way that doesn't incur size/abstraction
penalty. Some aspects of that are handled by modules, some probably
aren't. Overall, the "in size" "physical design" aspect of having to
state the types of members in a class definition can indeed be problematic
in the sense that you have to commit to the types in a class definition
regardless of whether the types are ever used by clients. There are
work-arounds(*) for that, so the question is whether language support
for such a facility is truly necessary and/or worth the cost of it.

(*) The work-arounds aren't necessarily easy - having a data buffer
in a class and having pimpl-side implementations know the actual
type of objects stored in such a buffer require proper alignment
and proper lifetime handling, which can be non-trivial.

[mobi phil]

Post by mobi phil

Post by mobiphil

Post by mobi phil
While I do not know exactly the formal procedure, I do not think it is
worth the effort to write a formal paper unless some of the honorable
members of the honorable committee would show minimum interest to ever
consider it. The reason of the brainstorming is to find together

weaknesses

Post by mobiphil

Post by mobi phil
of the model and solutions for them.

I agree

As far as I understand it, the idea deals with being able to express
interface-implementation splits in a way that doesn't incur
size/abstraction
penalty. Some aspects of that are handled by modules, some probably
aren't.

If you change the class definition, you need to recompile module +
everything depends on module.
If you split the stuff into to, you need to recompile only if you change
the real interface towards the rest of the world.
In my experience you spend 90% of time doing useless recompiles just
because you need to ad a damm' private method. If it is private why do I
need to recompile billions of lines of code in client code?

While this is not the only benefit the proposal would bring, it is the only
one that would have some way to deal with modules. Unfortunately though the
same benefit could not be achieved by modules.

Post by mobi phil
Overall, the "in size" "physical design" aspect of having to
state the types of members in a class definition can indeed be problematic
in the sense that you have to commit to the types in a class definition
regardless of whether the types are ever used by clients. There are
work-arounds(*) for that, so the question is whether language support
for such a facility is truly necessary and/or worth the cost of it.
(*) The work-arounds aren't necessarily easy - having a data buffer
in a class and having pimpl-side implementations know the actual
type of objects stored in such a buffer require proper alignment
and proper lifetime handling, which can be non-trivial.

Of course there are work-arounds. Pimpl would be just an annoying hack that
would make inheritance difficult. Alignment and similar problems should be
the compilers problem.

"there is a workaround", pattern is often used, as most of the cases there
is a workaround. There is a workaround for any aspect for which C++ tries
to prove that it is inferior to C, including object oriented programming,
type safety, templates etc. You can implement dynamic type checking and you
will be on the safe side. You can implement constructs that mimic templates
at a very high cost, including some trivial text based templates etc. etc.
For everything there is a workaround. Unfortunately most of the cases it
turns out that work-arounds are ugly hacks with serious negative
consequences.

For me the equation is simple:
-> how many people/projects are using pimple?
-> why are they using pimple?
-> what is overhead to implement pimple for each class?
-> how much this overhead compromises the readibility of the code?
-> how many people/projects avoided pimple, just because pimple is ugly and
preferred to discard advantages concerning the binary interface

[Vicente J. Botet Escriba]

Post by mobi phil

Post by mobiphil

Post by mobi phil
While I do not know exactly the formal procedure, I do not

think it is

Post by mobiphil

Post by mobi phil
worth the effort to write a formal paper unless some of the

honorable

Post by mobiphil

Post by mobi phil
members of the honorable committee would show minimum interest

to ever

Post by mobiphil

Post by mobi phil
consider it. The reason of the brainstorming is to find

together weaknesses

Post by mobiphil

Post by mobi phil
of the model and solutions for them.

I agree

As far as I understand it, the idea deals with being able to express
interface-implementation splits in a way that doesn't incur size/abstraction
penalty. Some aspects of that are handled by modules, some probably
aren't.
If you change the class definition, you need to recompile module +
everything depends on module.

I don't know well the module proposal, but I would expect that only
everything that depends on the public part of the module.

Post by mobi phil
If you split the stuff into to, you need to recompile only if you
change the real interface towards the rest of the world.

I would expect modules would bring this.

Post by mobi phil
In my experience you spend 90% of time doing useless recompiles just
because you need to ad a damm' private method. If it is private why do
I need to recompile billions of lines of code in client code?

Agreed, there is no need.

Post by mobi phil
While this is not the only benefit the proposal would bring, it is the
only one that would have some way to deal with modules. Unfortunately
though the same benefit could not be achieved by modules.

Why? What is missing in modules so that we can achieve it?

Post by mobi phil
Overall, the "in size" "physical design" aspect of having to
state the types of members in a class definition can indeed be problematic
in the sense that you have to commit to the types in a class definition
regardless of whether the types are ever used by clients. There are
work-arounds(*) for that, so the question is whether language support
for such a facility is truly necessary and/or worth the cost of it.
(*) The work-arounds aren't necessarily easy - having a data buffer
in a class and having pimpl-side implementations know the actual
type of objects stored in such a buffer require proper alignment
and proper lifetime handling, which can be non-trivial.
Of course there are work-arounds. Pimpl would be just an annoying hack
that would make inheritance difficult. Alignment and similar problems
should be the compilers problem.
"there is a workaround", pattern is often used, as most of the cases
there is a workaround. There is a workaround for any aspect for which
C++ tries to prove that it is inferior to C,

Don't forget that C++ includes C, so that anything you can do with C you
can do with C++.

Post by mobi phil
including object oriented programming, type safety, templates etc. You
can implement dynamic type checking and you will be on the safe side.
You can implement constructs that mimic templates at a very high cost,
including some trivial text based templates etc. etc. For everything
there is a workaround. Unfortunately most of the cases it turns out
that work-arounds are ugly hacks with serious negative consequences.
-> how many people/projects are using pimple?
-> why are they using pimple?
-> what is overhead to implement pimple for each class?
-> how much this overhead compromises the readibility of the code?
-> how many people/projects avoided pimple, just because pimple is
ugly and preferred to discard advantages concerning the binary interface

I don't think pimpl is the idiom needed to split the back and the white
parts of a class. IMHO, it is more the backdoor idiom that is needed.
The backdoor is the black part of your proposal.

C.hpp
-----
class C {
public:
// public member functions declaration
protected:
// protected member functions declaration
private:
aligned_storage<N>storage;
friend struct black;
};


C.cpp
-----

struct C::black {
C& that_;
black(C& that) : that_(that) {}
struct private_ {... };
static_assert(sizeof(private_) == N, "Public and private sizes don't
match");
private_& pdata() { return *reinterpret_cast<private_*>(that_.storage);}

// Add here any private static or non static member
};

int C::f() {
// use of the C::black(*this) for any private access, either for data
or private functions
// use of C::black:: for any internal type or static member
}

My questions are:
* how many people/projects having compile time issues (millions of
lines) don't use this backdoor idiom?
* why?

I'm all for a future C++ in which we don't need this idiom, and I expect
that the Module proposal would provide it.

Vicente

[mobi phil]

Post by mobi phil
If you split the stuff into to, you need to recompile only if you change
the real interface towards the rest of the world.
I would expect modules would bring this.

well, while I do not know in depth the modules proposal, I would rather
bet, that it will not be able to do the impossible. I have doubts that it
would be so clever, that will not force clients to recompile once you add
the declaration of a private method.

Post by mobi phil
While this is not the only benefit the proposal would bring, it is the
only one that would have some way to deal with modules. Unfortunately
though the same benefit could not be achieved by modules.
Why? What is missing in modules so that we can achieve it?

see above: IMHO modules will not be able to deal with
-> do not recompile client code if you add private method
-> do not recompile if you change class internal data
-> recompile only if the interface of a class changes, in best case if the
method definition that client code depends on changes.

Please note that this black-white proposal does not mean that all classes
have to behave this "dynamic" way.

Post by mobi phil
Of course there are work-arounds. Pimpl would be just an annoying hack
that would make inheritance difficult. Alignment and similar problems
should be the compilers problem.
"there is a workaround", pattern is often used, as most of the cases
there is a workaround. There is a workaround for any aspect for which C++
tries to prove that it is inferior to C,
Don't forget that C++ includes C, so that anything you can do with C you
can do with C++.

Thanks for telling me that... really forgot :) ... well trying to be a bit
sarcastic here ... Then please read it: "with the C core features"

Post by mobi phil
including object oriented programming, type safety, templates etc. You
can implement dynamic type checking and you will be on the safe side. You
can implement constructs that mimic templates at a very high cost,
including some trivial text based templates etc. etc. For everything there
is a workaround. Unfortunately most of the cases it turns out that
work-arounds are ugly hacks with serious negative consequences.
-> how many people/projects are using pimple?
-> why are they using pimple?
-> what is overhead to implement pimple for each class?
-> how much this overhead compromises the readibility of the code?
-> how many people/projects avoided pimple, just because pimple is ugly
and preferred to discard advantages concerning the binary interface
I don't think pimpl is the idiom needed to split the back and the
white parts of a class. IMHO, it is more the backdoor idiom that is needed.
The backdoor is the black part of your proposal.
C.hpp
-----
class C {
// public member functions declaration
// protected member functions declaration
aligned_storage<N>storage;
friend struct black;
};
C.cpp
-----
struct C::black {
C& that_;
black(C& that) : that_(that) {}
struct private_ {... };
static_assert(sizeof(private_) == N, "Public and private sizes don't
match");
private_& pdata() { return *reinterpret_cast<private_*>(that_.storage);}
// Add here any private static or non static member
};
int C::f() {
// use of the C::black(*this) for any private access, either for data
or private functions
// use of C::black:: for any internal type or static member
}
* how many people/projects having compile time issues (millions of lines)
don't use this backdoor idiom?
* why?
I'm all for a future C++ in which we don't need this idiom, and I expect
that the Module proposal would provide it.

while you remove the problem of pimple, you introduce sthg. else, which my
proposal tries to solve. You are strongly depending on this N. Too loosen a
bit this dependency you could make at the beginning this N enough big, but
that would lead to waste of memory.

My proposal covers the fact that this "N" is practically replaced at
runtime, with the real size.

[Vicente J. Botet Escriba]

Post by Vicente J. Botet Escriba

Post by mobi phil
If you split the stuff into to, you need to recompile only if you
change the real interface towards the rest of the world.

I would expect modules would bring this.
well, while I do not know in depth the modules proposal, I would
rather bet, that it will not be able to do the impossible. I have
doubts that it would be so clever, that will not force clients to
recompile once you add the declaration of a private method.

IMO, this should be possible and so it is a QOI. Only if the
implementers consider that this wouldn't be possible or too complex we
would need to adapt the proposal.

Post by Vicente J. Botet Escriba

Post by mobi phil
While this is not the only benefit the proposal would bring, it
is the only one that would have some way to deal with modules.
Unfortunately though the same benefit could not be achieved by
modules.

Why? What is missing in modules so that we can achieve it?
see above: IMHO modules will not be able to deal with
-> do not recompile client code if you add private method
-> do not recompile if you change class internal data
-> recompile only if the interface of a class changes, in best case if
the method definition that client code depends on changes.

IMHO, it should be easier to ensure that the module proposal is able to
do it that adding your alternative approach.

Post by Vicente J. Botet Escriba
Please note that this black-white proposal does not mean that all
classes have to behave this "dynamic" way.

I don't see anything dynamic in the behavior you have defined.

Post by Vicente J. Botet Escriba
<snip>

Post by mobi phil
-> how many people/projects are using pimple?
-> why are they using pimple?
-> what is overhead to implement pimple for each class?
-> how much this overhead compromises the readibility of the code?
-> how many people/projects avoided pimple, just because pimple
is ugly and preferred to discard advantages concerning the binary
interface

I don't think pimpl is the idiom needed to split the back and the
white parts of a class. IMHO, it is more the backdoor idiom that
is needed. The backdoor is the black part of your proposal.
<snip>
I'm all for a future C++ in which we don't need this idiom, and I
expect that the Module proposal would provide it.
while you remove the problem of pimple, you introduce sthg. else,
which my proposal tries to solve. You are strongly depending on this
N. Too loosen a bit this dependency you could make at the beginning
this N enough big, but that would lead to waste of memory.

There is not big or small, just are equals or not. This N, should be
provided by the compiler from the module definition as well as all other
public and protected functions.

Post by Vicente J. Botet Escriba
My proposal covers the fact that this "N" is practically replaced at
runtime, with the real size.

We don't need it at run-time, but at compile time, and IMO, the Module
proposal should be such that such implementation is possible.

After reading the module proposal, it seams that, they are not
addressing this problem. I suggest you to join the Module ML and request
for the requirements you are looking for (not the solutions). Maybe your
white/black split could be the solution.

c++std-***@accu.org

Vicente

[mobi phil]

Post by mobi phil
If you split the stuff into to, you need to recompile only if you

Post by mobi phil
change the real interface towards the rest of the world.
I would expect modules would bring this.

well, while I do not know in depth the modules proposal, I would rather
bet, that it will not be able to do the impossible. I have doubts that it
would be so clever, that will not force clients to recompile once you add
the declaration of a private method.
IMO, this should be possible and so it is a QOI. Only if the implementers
consider that this wouldn't be possible or too complex we would need to
adapt the proposal.

I do not pretend that I know the proposal, I have some basic idea of how
clang is doing it. Without going into the details I think there are two
things that needs to be looked at separated:
-> if the header changes, the module that depends on it needs to be rebuilt.
-> the way at least clang approached it, you still need to work with
makefile dependencies.

If you changed the header, the timestamp, checksum, or whatsoever way you
would implement dependencies, the source file that depends on the header in
question will have to be rebuilt. This must be true even in case where you
add a private method, unless a special file is generated for dependencies
that reflect only really "public" stuff from headers.

Post by mobi phil
While this is not the only benefit the proposal would bring, it is the

Post by mobi phil
only one that would have some way to deal with modules. Unfortunately
though the same benefit could not be achieved by modules.
Why? What is missing in modules so that we can achieve it?

see above: IMHO modules will not be able to deal with
-> do not recompile client code if you add private method
-> do not recompile if you change class internal data
-> recompile only if the interface of a class changes, in best case if the
method definition that client code depends on changes.
IMHO, it should be easier to ensure that the module proposal is able to do
it that adding your alternative approach.

While reducing dependencies is one of the goals of my approach/proposal I
really doubt that handling private data changes in headers will be delt
with modules. For simple reason that any source file that depends on the
definition of the class (due to sizeof()) will need to recompile once the
size of the class changes. In my proposal this dependency is postponed till
runtime.

Post by mobi phil
Please note that this black-white proposal does not mean that all classes
have to behave this "dynamic" way.
I don't see anything dynamic in the behavior you have defined.

Well, do not wonder, I have the impression that you did not understand the
main idea.
Did you carefully read the original post? Sorry, but I doubt

Let's rewrite the reasoning with different perspective
-> What do we want?
->>> to really hide private part of a class
--->>>> private methods
--->>>> private data
-> What is the main problem with this?
--->>>> private methods: not really.. they are forced to be declared in the
class declaration, without any reason, so no problem to remove
--->>>> private data: well... this is the problem, as "client" classes need
to know the size of the class

---> solution: make any local/global/heap allocation depending on a
"global" variable that will be known at dynamic linking

while this would open doors for other dynamic things, you already see a
little "dynamic" here.

while you remove the problem of pimple, you introduce sthg. else, which my

Post by mobi phil
proposal tries to solve. You are strongly depending on this N. Too loosen a
bit this dependency you could make at the beginning this N enough big, but
that would lead to waste of memory.
There is not big or small, just are equals or not. This N, should be
provided by the compiler from the module definition as well as all other
public and protected functions.
My proposal covers the fact that this "N" is practically replaced at
runtime, with the real size.
We don't need it at run-time, but at compile time, and IMO, the Module
proposal should be such that such implementation is possible.

Well, while do not know who are the "we" I need it and I see tha major
advantage. Whatsoever way you define N at compile time, you will force
everybody who depends on N to rebuild once you change the size of the
implementation which is exactly the thing I want to achieve among others.

Another point of this "dynamic" classes is that you could use C++ classes
from dynamic libraries without any pimpl overhead.

Post by mobi phil
After reading the module proposal, it seams that, they are not addressing
this problem. I suggest you to join the Module ML and request for the
requirements you are looking for (not the solutions). Maybe your
white/black split could be the solution.
No problem to help there, but personally I fail to see that this could be

done with modules...

[Thiago Macieira]

Post by mobi phil
If you changed the header, the timestamp, checksum, or whatsoever way you
would implement dependencies, the source file that depends on the header in
question will have to be rebuilt. This must be true even in case where you
add a private method, unless a special file is generated for dependencies
that reflect only really "public" stuff from headers.

Unless, of course, that class has friends. In which case, even private changes
will require rebuilding.

Post by mobi phil
While reducing dependencies is one of the goals of my approach/proposal I
really doubt that handling private data changes in headers will be delt
with modules. For simple reason that any source file that depends on the
definition of the class (due to sizeof()) will need to recompile once the
size of the class changes. In my proposal this dependency is postponed till
runtime.

Your idea, not proposal. You need to do a little more work to make it a
proposal, including writing a paper and presenting it in a committee meeting.
Or convincing someone to do it for you.

Post by mobi phil

Post by mobi phil
After reading the module proposal, it seams that, they are not addressing
this problem. I suggest you to join the Module ML and request for the
requirements you are looking for (not the solutions). Maybe your
white/black split could be the solution.
No problem to help there, but personally I fail to see that this could be

done with modules...

Are you saying that modules would make it impossible to implement your
suggestion?

Because if you aren't saying that, then you're missing the point. We're not
saying that modules is doing what you suggested. We are saying that you should
join the modules discussion so that your solution gets worked into the modules
proposal.
--
Thiago Macieira - thiago (AT) macieira.info - thiago (AT) kde.org
Software Architect - Intel Open Source Technology Center
PGP/GPG: 0x6EF45358; fingerprint:
E067 918B B660 DBD1 105C 966C 33F5 F005 6EF4 5358

[mobi phil]

Post by Thiago Macieira

Post by mobi phil
If you changed the header, the timestamp, checksum, or whatsoever way you
would implement dependencies, the source file that depends on the header

in

Post by mobi phil
question will have to be rebuilt. This must be true even in case where

you

Post by mobi phil
add a private method, unless a special file is generated for dependencies
that reflect only really "public" stuff from headers.

Unless, of course, that class has friends. In which case, even private changes
will require rebuilding.

Friendship as always should be sometimes reconsidered. If not really
necessary for performance reason, just use getters/setters.
But if really necessary and most of the time friends like to be together,
most of the time they are close to each other.
Such friendship would be declared in the black, and friends may/should know
about blacks, but they should not propagate this dependency.

inside A.h

white A { /* white is the real iterface, could be called "class iface"

}

inside A.cpp

black A {


friend black F;
}

inside F.cpp
#include "A.cpp"
#include "F.h"

the black of A could also be factored into A.black.h, that would be
included by both A.cpp and F.cpp

Post by Thiago Macieira
While reducing dependencies is one of the goals of my approach/proposal I

Post by mobi phil
really doubt that handling private data changes in headers will be delt
with modules. For simple reason that any source file that depends on the
definition of the class (due to sizeof()) will need to recompile once the
size of the class changes. In my proposal this dependency is postponed

till

Post by mobi phil
runtime.

Your idea, not proposal. You need to do a little more work to make it a
proposal, including writing a paper and presenting it in a committee meeting.
Or convincing someone to do it for you.

Let's not get lost in words. It is a proposal, though it is not a formal
one. It was not just an idea, I thought about at coffee time. Have it in my
mind for a while and though about certain aspects. While I wanted to
implement it in pure C with macros, gave up. Though could reformulate the
original post, do not feel at the moment it would be worth to go into a
deep analyzis if the added value is not recognized.

Post by Thiago Macieira

Post by mobi phil

Post by Vicente J. Botet Escriba
After reading the module proposal, it seams that, they are not

addressing

Post by mobi phil

Post by Vicente J. Botet Escriba
this problem. I suggest you to join the Module ML and request for the
requirements you are looking for (not the solutions). Maybe your
white/black split could be the solution.
No problem to help there, but personally I fail to see that this could

be

Post by mobi phil
done with modules...

Are you saying that modules would make it impossible to implement your

suggestion?
Because if you aren't saying that, then you're missing the point. We're not

Post by Thiago Macieira
saying that modules is doing what you suggested. We are saying that you should
join the modules discussion so that your solution gets worked into the modules
proposal.

To avoid further confusion: I am not saying anything else than that I have
strong doubts that this could be worked in some way into modules if you do
not add the necessary language extension. Once you added those extensions,
should be treated as a separate story as it involves mainly non-module
related changes.

[Thiago Macieira]

Post by mobi phil

Post by Thiago Macieira
Unless, of course, that class has friends. In which case, even private changes
will require rebuilding.

Friendship as always should be sometimes reconsidered. If not really
necessary for performance reason, just use getters/setters.

That's advice. Unless forbid the practice, the advice can be ignored and you
have to deal with it.

It's irrelevant whether it's a bad practice or not.

Post by mobi phil
But if really necessary and most of the time friends like to be together,
most of the time they are close to each other.
Such friendship would be declared in the black, and friends may/should know
about blacks, but they should not propagate this dependency.
inside A.h
white A { /* white is the real iterface, could be called "class iface"
}
inside A.cpp
black A {
friend black F;
}
inside F.cpp
#include "A.cpp"
#include "F.h"
the black of A could also be factored into A.black.h, that would be
included by both A.cpp and F.cpp

Sounds like advice. Therefore, it can be ignored.

Post by mobi phil
Let's not get lost in words. It is a proposal, though it is not a formal
one. It was not just an idea, I thought about at coffee time. Have it in my
mind for a while and though about certain aspects. While I wanted to
implement it in pure C with macros, gave up. Though could reformulate the
original post, do not feel at the moment it would be worth to go into a
deep analyzis if the added value is not recognized.

What recognition are you expecting before you go further?

Post by mobi phil

Post by Thiago Macieira
Are you saying that modules would make it impossible to implement your
suggestion?
Because if you aren't saying that, then you're missing the point. We're not
saying that modules is doing what you suggested. We are saying that you should
join the modules discussion so that your solution gets worked into the modules
proposal.

To avoid further confusion: I am not saying anything else than that I have
strong doubts that this could be worked in some way into modules if you do
not add the necessary language extension. Once you added those extensions,
should be treated as a separate story as it involves mainly non-module
related changes.

Understood.

I still recommend talking to the people doing modules because they may have
other points to bring up and they're the right people to help you with your
suggestion.
--
Thiago Macieira - thiago (AT) macieira.info - thiago (AT) kde.org
Software Architect - Intel Open Source Technology Center
PGP/GPG: 0x6EF45358; fingerprint:
E067 918B B660 DBD1 105C 966C 33F5 F005 6EF4 5358

[mobi phil]

Post by Thiago Macieira

Post by mobi phil

Post by Thiago Macieira
Unless, of course, that class has friends. In which case, even private changes
will require rebuilding.

Friendship as always should be sometimes reconsidered. If not really
necessary for performance reason, just use getters/setters.

That's advice. Unless forbid the practice, the advice can be ignored and you
have to deal with it.
It's irrelevant whether it's a bad practice or not.

while this new construct has to have a different name than "class" or
another keyword attached to it, it is straightforward to turn the above
into a rule by forbidding friend keyword inside the body of with the white.

Post by Thiago Macieira
Though could reformulate the

Post by mobi phil
original post, do not feel at the moment it would be worth to go into a
deep analyzis if the added value is not recognized.

What recognition are you expecting before you go further?

Well, so far you as QT "guy" (I understand), you did not seem to show any
sign that it would be valuable in a framework like QT (obviously without
thinking about rewriting parts of it, to make this used). Obviously this
does not mean that I do not see that value.

Post by Thiago Macieira
Understood.
I still recommend talking to the people doing modules because they may have
other points to bring up and they're the right people to help you with your
suggestion.

Will try to do so, just that it needs first go through those papers and not
too much time for it.

[Thiago Macieira]

Post by mobi phil

Post by Thiago Macieira
What recognition are you expecting before you go further?

Well, so far you as QT "guy" (I understand), you did not seem to show any
sign that it would be valuable in a framework like QT (obviously without
thinking about rewriting parts of it, to make this used). Obviously this
does not mean that I do not see that value.

I have two qualms about the proposal: one, which we discussed, is the runtime
overhead which may be too great, greater than the use of d pointers (pimpl).
The other is that I can't use for a long time. Let me put it this way: if this
showed up for C++17, we might be able to start relying on it in Qt by 2025.

Other than that, the idea is interesting.

If you want more feedback, write a paper.
--
Thiago Macieira - thiago (AT) macieira.info - thiago (AT) kde.org
Software Architect - Intel Open Source Technology Center
PGP/GPG: 0x6EF45358; fingerprint:
E067 918B B660 DBD1 105C 966C 33F5 F005 6EF4 5358

[mobi phil]

Post by Thiago Macieira
I have two qualms about the proposal: one, which we discussed, is the runtime
overhead which may be too great, greater than the use of d pointers (pimpl).

I do not think this would be the case. For accessing data members that are
in the pimpl or back through getters/setters, the overhead would be equal,
that is a function call. For data members that are in the "white", indeed
there is an overhead compared to normal member access, but this overhead
would not be greater than accessing "black" members.
For both "black" and "white" members (well, both for real public ones from
white and the one's that are accessed through getters/setters from black)
one could think about another language extension I would call it runtime
inline-ing. Given a pair of setter/getter or "white" public data member,
the compiler would generate placeholder code and the dynamic loader would
patch it with the correct de-referencing of the member based on the "final"
layout of the class. This mechanism will trade code space for this patching
versus efficiency of patching/replacement.

Overhead for creating objects would be in worst case equal with pimpl.

Post by Thiago Macieira
The other is that I can't use for a long time. Let me put it this way: if this
showed up for C++17, we might be able to start relying on it in Qt by 2025.

Well, tend to agree, but this sad law of reality applies to any other
proposals.

Other than that, the idea is interesting.
If you want more feedback, write a paper.
Will try to write then a first version of the paper. At the end of the day
it does not need to pretend it would cover all the details about the impact
on the future of the humanity :)

[mobi phil]

is this the paper about modules?

http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2006/n2073.pdf

did skim it, but:
-> not too much thing is clearly proposed as standard in terms of templates
instantiation which in my opinion takes most of compilation times
-> does not say anything about recompilation dependencies when changing the
data members thus size of classes...

will go later to their mailing list, to form an idea where they are

[Klaim - Joël Lamotte]

Post by mobi phil
is this the paper about modules?
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2006/n2073.pdf

There have been several proposals and updates since this one (see the year
in the url).

The last module proposal is from microsoft (they have a private
implementation apparently):
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2014/n4047.pdf

It's similar to the one implemented in clang, with a few differences. See
http://clang.llvm.org/docs/Modules.html

[mobi phil]

thanks for the correct link.

-> I was investigating clang modules implementation few weeks ago.
Unfortunately did not gain anything in terms of compilation time
improvements for some source code that was heavily depending on heavy boost
templates. It does not support implicit template instantiations to be cache
in the compiled modules, which for me is the most important thing. Would
love to see instant compilation for a file that includes few headers that
contain heavy templates. Maybe I failed to grasp how to do it and raised
the question on clang dev-list, but did not get till today a clear answer.
Would love to see somebody showing me real compilation time gain when
building such. But this is not the topic of this thread.

-> went through the latest proposal:
->> The proposal luckily makes reference to implicit template instantiation
caching.
->> Do not like that they are convinced to throw away the preprocesor
without creating something in place
->> related to my current proposal (see also the quote from the paper
below), they touch the subject about "changes to private member" but the
seem not to want to adventure by coming up with any solution.

from the proposal
"An occasionally vexing rule of standard C++ is that protection controls
access,
not visibility. E.g. a private member of a class is visible to, but not
accessible to
non-member entities. In particular, any change to a private member of a
class is
likely to trigger re-processing of any translation unit that depends on
that class’s
definition even if the change does not affect the validity of dependent
units. It
is tempting to solve that problem with a module system. However, having two
distinct sets of rules (visibility and accessibility) for class members
strikes us as
undesirable and potentially fertile source of confusion. Furthermore, we
want to
support mass-migration of existing codes to modules without programmers
having
to worry about class member name lookup rules: if you understand those rules
today, then you do not have to learn new rules when you move to modules and
you
do not have to worry about how the classes you consume are provided (via
modules
or non-modules).
That being said, we believe the visibility vs. accessibility issue is a
problem that
should be solved by an orthogonal language construct, irrespectively of
whether a
class is defined in a module interface declaration or in an ordinary
translation unit.

[Vicente J. Botet Escriba]

Post by mobi phil
thanks for the correct link.
-> I was investigating clang modules implementation few weeks ago.
Unfortunately did not gain anything in terms of compilation time
improvements for some source code that was heavily depending on heavy
boost templates. It does not support implicit template instantiations
to be cache in the compiled modules, which for me is the most
important thing. Would love to see instant compilation for a file that
includes few headers that contain heavy templates. Maybe I failed to
grasp how to do it and raised the question on clang dev-list, but did
not get till today a clear answer. Would love to see somebody showing
me real compilation time gain when building such. But this is not the
topic of this thread.

Agreed. However the goal is the same: reduce compile time.

Post by mobi phil
->> The proposal luckily makes reference to implicit template
instantiation caching.
->> Do not like that they are convinced to throw away the preprocesor
without creating something in place
->> related to my current proposal (see also the quote from the paper
below), they touch the subject about "changes to private member" but
the seem not to want to adventure by coming up with any solution.
from the proposal
"An occasionally vexing rule of standard C++ is that protection
controls access,
not visibility. E.g. a private member of a class is visible to, but
not accessible to
non-member entities. In particular, any change to a private member of
a class is
likely to trigger re-processing of any translation unit that depends
on that class’s
definition even if the change does not affect the validity of
dependent units. It
is tempting to solve that problem with a module system. However, having two
distinct sets of rules (visibility and accessibility) for class
members strikes us as
undesirable and potentially fertile source of confusion. Furthermore,
we want to
support mass-migration of existing codes to modules without
programmers having
to worry about class member name lookup rules: if you understand those rules
today, then you do not have to learn new rules when you move to
modules and you
do not have to worry about how the classes you consume are provided
(via modules
or non-modules).
That being said, we believe the visibility vs. accessibility issue is
a problem that
should be solved by an orthogonal language construct, irrespectively
of whether a
class is defined in a module interface declaration or in an ordinary
translation unit.

Thanks for the quote. It seems I was wrong, and that the module proposal
would not take care of the visibility issues.

I believe that you have here the go from the honorable people you were
looking for, at least for the problem to be solved. I'm sure the same
people would be interested in discussing possible solutions.

I will add this quoting the proposal:
"Runtime Performance
Moving an existing code to a brave new module world, or writing new
codes with modules, should not in any way degrade its runtime
performance characteristics. In particular, we do not seek a module
system requiring a compiler to perform automatic “boxing” of object
representation (exposed in class private members) –in attempts to
reducing re-compilation– via opaque data pointers a la ` pImpl idiom."


Respect to the visibility issue, I would like to preserve Runtime
Performance also: Moving an existing code to a brave new visibility
aware world, or writing new codes with visibility concerns, should not
in any way degrade its runtime performance characteristics."

I'm not saying that I'm against any visibility solution that doesn't
preserve runtime performance, just that the performance aspect must be
considered.

Vicente

[mobi phil]

Post by Vicente J. Botet Escriba
"Runtime Performance
Moving an existing code to a brave new module world, or writing new codes
with modules, should not in any way degrade its runtime performance
characteristics. In particular, we do not seek a module system requiring a
compiler to perform automatic “boxing” of object representation (exposed in
class private members) –in attempts to reducing re-compilation– via opaque
data pointers a la ` pImpl idiom."
Respect to the visibility issue, I would like to preserve Runtime
Performance also: Moving an existing code to a brave new visibility aware
world, or writing new codes with visibility concerns, should not in any way
degrade its runtime performance characteristics."
I'm not saying that I'm against any visibility solution that doesn't
preserve runtime performance, just that the performance aspect must be
considered.

Well, while it would be annoying to have a general solution where private
data would be automatically moved to some pimpl one has to have the freedom
to do so. Some applications may suffer from this little performance penalty
of the black/white approach, but as described earlier this would be the
same overhead as a non-inlined pair of setter/getter. You are trading
something against something.

While non-inlined setters/getters would be a performance penalty for both
actual approach and black/white, maybe one could think of some link time or
dynamic link time inlining solution, where a certain code binary code
pattern could be replaced with the "real implementation".

[mobi phil]

FYI: some of you for sure know about llvm-lto. It could solve the overhead
of getter/setters or with other words how offsets would be solved.

llvm/clang has llvm-lto and the gold plugin that can inline functions at
linking time. The setters/getters defined in the black and containing the
correct member offset information could be inlined at link time. It seems
that there would be no runtime access overhead for data members defined in
"black".

I assume that this trick with lto(=inline) could be used also for pimpl
related call overhead.

[Chris Gary]

This is an attractive alternative to pointer-based PIMPL, though it would
require sweeping modifications to be made across every compliant
loader/toolset.

That said, I still like it.

However, it doesn't seem to address re-using the same interface for
multiple implementations simultaneously (exposing "new" through the
interface layer constrains it to a single implementation).

Another issue is version discrepancy: What metadata must be stored in
external module containing all the "black" in order to ensure that the
addition of new members to that layer do not cause undefined behavior when
linked against an older "white" that doesn't know about them? There needs
to be a way to verify a new "black" against an already compiled "white"
during compilation, which seems to indicate the need for a large amount of
metadata...

These concerns stem from a number of problems I've been facing lately:

Say we need to display the same dataset on a workstation monitor and
through a projector.

The monitor is connected through an Nvidia adapter, and the projector
through an old ATI.

We have to create one rendering context per adapter, as moving a window
from the workstation display to the projector will crash or result in
garbage (unless all rendering is done with GDI).

Both contexts have different feature sets resulting in different "black"
layers, however the "white" layer looks the same for both.

The obvious solution is PIMPL, though there is a lot of work involved in
maintaining the parallel sets of types required to realize this.
After a small change in the interface, resolving version discrepancies for
every affected module slows development to a crawl.

The biggest time-waster is a new "black" failing its tests after
compilation.

[Chris Gary]

Answering my first question: Just have the "black" layer return a pointer,
which brings us back to PIMPL.

The real power of this proposal would be a standardized way to validate an
implementation against a compiled interface.

Post by Chris Gary
This is an attractive alternative to pointer-based PIMPL, though it would
require sweeping modifications to be made across every compliant
loader/toolset.
That said, I still like it.
However, it doesn't seem to address re-using the same interface for
multiple implementations simultaneously (exposing "new" through the
interface layer constrains it to a single implementation).
*snip*

[mobi phil]

Post by Chris Gary
This is an attractive alternative to pointer-based PIMPL, though it would
require sweeping modifications to be made across every compliant
loader/toolset.

yes, but as usual, the classes without the additional keyword are supposed
to behave as earlier.

Post by Chris Gary
That said, I still like it.

happy to hear that

However, it doesn't seem to address re-using the same interface for

Post by Chris Gary
multiple implementations simultaneously (exposing "new" through the
interface layer constrains it to a single implementation).

Does not address multiple implementations as you could have multiple
implementations exactly like for a usual cpp file that implements it's
header. This can be seen bad or good. You may provide for linking one or
other version, depending on your intentions.

Post by Chris Gary
Another issue is version discrepancy: What metadata must be stored in
external module containing all the "black" in order to ensure that the
addition of new members to that layer do not cause undefined behavior when
linked against an older "white" that doesn't know about them? There needs
to be a way to verify a new "black" against an already compiled "white"
during compilation, which seems to indicate the need for a large amount of
metadata...

Not sure if I completely understand your concern, but let me add some
thoughts. Indeed I did not advise if there should be one black for the same
white interface. Allowing different "cumulative" blacks in different files
for the same white would allow much more flexibility at a very high cost
(complexity). Thus from the beginning I would discard that option, so only
one black for each white.
Also keep in mind, like above even now you can compile "foo.cpp" against
"foo.h" but "bar.cpp" against an older version of "foo.h". There may be
serious object misalligment that may cause crash. Thus similar care must be
taken for building, but build (dependency etc.) management discipline is
out of scope.
On the other side this black/white story would reduce such dangers, as
class layout information would be moved to only one place and that is the
black.

Post by Chris Gary
Say we need to display the same dataset on a workstation monitor and
through a projector.
The monitor is connected through an Nvidia adapter, and the projector
through an old ATI.
We have to create one rendering context per adapter, as moving a window
from the workstation display to the projector will crash or result in
garbage (unless all rendering is done with GDI).
Both contexts have different feature sets resulting in different "black"
layers, however the "white" layer looks the same for both.

Do not have too much knowledge here, but my general idea abut it was that
such implementation details about rendering should be done in the graphics
card driver, and the user space program should not be aware if it is
displaying on one or other monitor.
Nevertheless this would go a bit out of the scope of the proposal.
Only related thing I can comment on is that at run-time there would be only
one black. Design patters like polymorphism/strategy would be implemented
like before.

Post by Chris Gary
The obvious solution is PIMPL, though there is a lot of work involved in
maintaining the parallel sets of types required to realize this.

Take it like that this black/white is at the end of the day PIMPL without
the pointer overhead and parallel set of types.

Post by Chris Gary
After a small change in the interface, resolving version discrepancies for
every affected module slows development to a crawl.

Still not clear. If you change the interface, your dependency system should
rebuild everything. One of the gains with with black/white is that you
reduce this rebuild risk to minimum.

Conclusion: I am afraid did not understand clearly your concerns. Perhaps
you did not understand completely the concept?

[Chris Gary]

If you don't mind me horribly abusing the terminology here...

From what I gather: "white" is the interface and "black" is the
implementation (duh). Instead of the pointer-to-PIMPL, this would work
kinda like partial classes in C#: The loader fills out offset/size details
at load time.

The display example is just one of several (Nvidia and ATI crap don't get
along, so everything breaks, but that's beside the point).

Rebuilding isn't the problem: Its validating changes that have to be made
on a per-implementation basis.

The problem, in general, is that there are numerous interfaces that the
"core" application is meant to work with, and those interfaces are backed
by implementations known only at runtime (plugins).

This is why I asked about re-using interfaces: All the "whites" would be in
a central codebase (in some cases, compiled and distributed to end-users
who write their own plugins), and all the "blacks" would be implemented on
a per-plugin/component basis. Multiple plugins that back the same interface
would need to be loaded simultaneously.

The problem with the distribution of the "white" binary is that users will
need/want to add members to the "black" halves they are writing, creating
part of the "version discrepancy" issue I was describing (this also happens
when a new "white" binary is distributed that must work with all existing
"blacks").

In the display example, we'd need two different graphics backends: Usually
the small vs. large display or comparing OpenGL and D3D with synchronized
resources (CAD shapes w/ optimized representations for each).

D3D is very, very different from OpenGL, so the "black" half of either
would need to be updated by hand. Add to that OpenGL's abomination of an
"extension" system, and you have a swampland of implementations to fill in.

If your proposal is supposed to work statically, then I've completely
missed the boat (wouldn't surprise me, I've been looking for a way to
simplify these messes for years)...

[mobi phil]

Post by Chris Gary
If you don't mind me horribly abusing the terminology here...

any abuse should be accepted as long as it helps to converge ;)

First let's fix your terminology a bit to avoid confusion. You mentioned
core. This may create confusion. Let's better call it internal world of the
class (both black and white) and the external world (consumer) of the
class.

Let me also resume something which may be spread in the original post and
subseqeunt comments. The external world, or core as you call.

It is only the black that knows about the size and layout of the class
(black). The outside world can have access to this layout through
setters/getters.

Implementation detail cannot be ignored for a successful design of the
abstraction. So far two approaches were identified how to solve the problem
of data member offset. (the sizeof for new() is similar, let's ignore for
the moment)

1. without overhead of pointer dereference or function call, but less
flexible
2. with this overhead, either with pointers or function call, but more
flexible.

for 1.) the setters/getters would be inlined at link time with
setters/getters automatically generated by the compiler for black members.
for 2.) any setter/getter in the white, thus public data member would be
solved at runtime as:
-> a non inlined method implemented in the black
-> a pointer to a global offset table that contains the offset.

At the end the both implementation could be provided depending on the
usage.

In your example you seem to focus rather on 2.) which again:
-> has advantage that allows dynamic loading of blacks from shared
libraries, or different versions of them etc.
-> has the disadvantage that any member reference will "cost" an extra
pointer dereferencing that often may be expensive due to cache incoherence.


From what I gather: "white" is the interface and "black" is the

Post by Chris Gary
implementation (duh). Instead of the pointer-to-PIMPL, this would work
kinda like partial classes in C#: The loader fills out offset/size details
at load time.

se above... depends which implementation you would chose. If
setters/getters are inlined at link-time, the story at the end would be
exactly like with normal classes.

Post by Chris Gary
The display example is just one of several (Nvidia and ATI crap don't get
along, so everything breaks, but that's beside the point).

ok, let's forget this example, probably was not very relevant or confusing.

Post by Chris Gary
Rebuilding isn't the problem: Its validating changes that have to be made
on a per-implementation basis.

The problem, in general, is that there are numerous interfaces that the

Post by Chris Gary
"core" application is meant to work with, and those interfaces are backed
by implementations known only at runtime (plugins).

Again, based on the implementation one would choose, it can open the doors
for such mechanism like loading blacks as plugins.

Post by Chris Gary
This is why I asked about re-using interfaces: All the "whites" would be
in a central codebase (in some cases, compiled and distributed to end-users
who write their own plugins), and all the "blacks" would be implemented on
a per-plugin/component basis. Multiple plugins that back the same interface
would need to be loaded simultaneously.

I think I understand what is your point. But you go one step further. One
could the same time say you "abuse" the feature or you "bulid on top" of
the feature. In my original proposal, again, there is one black to one
white. What you say can be implemented if the offsets are implemented by
pointers or setters/getters. At the end of the day a black will be a set of
functions towards the external world, so you can see it as plugin. But this
is out of the scope of the original proposal. Though it is a strong point
to support it.

Post by Chris Gary
The problem with the distribution of the "white" binary is that users will
need/want to add members to the "black" halves they are writing, creating
part of the "version discrepancy" issue I was describing (this also happens
when a new "white" binary is distributed that must work with all existing
"blacks").
Att the end you may add anything to the black, even at runtime. What is

important is that any object of the black or subclass once crceated with a
certain version ( sizeof/new), should use the same implementation for
referencing members.

Post by Chris Gary
In the display example, we'd need two different graphics backends: Usually
the small vs. large display or comparing OpenGL and D3D with synchronized
resources (CAD shapes w/ optimized representations for each).
D3D is very, very different from OpenGL, so the "black" half of either
would need to be updated by hand. Add to that OpenGL's abomination of an
"extension" system, and you have a swampland of implementations to fill in.

again, this example is bit two complex and unclear. But if you are refering
to a software system where some OpenGL or D3D class layuot changes often
during development cycles and lot of code is depending on it, then the
proposed system would allow you to add/remove private methods and data
without having to rebuild tons of files depending on the class layout.

Post by Chris Gary
If your proposal is supposed to work statically, then I've completely
missed the boat (wouldn't surprise me, I've been looking for a way to
simplify these messes for years)...

again, see above. Probably both implementations should be supported (static
and dynamic)

it is on my TODO list to write a more formal version of the paper, where I
will take care to give more clear examples

[Chris Gary]

Based on your explanation, I have some suggestions.

To avoid introducing new CSK, you can re-use "virtual" and "override" for
this purpose.

For example:


class white button
{
public:

// Speculative data members
// (roughly equivalent to properties)
string title;
visual_style &style;
};

class black button
{
public:

string title;
HWND hWnd;
visual_style &style;

// Etc...
};



Change "white" to "virtual", and "black" to "override":

// Speculative interface
class virtual button
{
public:

string title;
visual_style &style;

// Note: We can still have virtual methods.
};

// Concrete definition
class override button
{
public:

// At least "string title" and "visual_style &style"
// must be declared. Anything else is free game.

string title;
HWND hWnd;
visual_style &style;
};



Accessing a member through a speculative interface would merely involve
fetching an offset from a vtable. Much like static binding of a virtual
method call, it is also possible to deduce a member offset statically if
the concrete definition is available.

Inheriting from a virtual class makes the result virtual and requires that
the override inherit from the same bases (possibly in the same order). It
might be reasonable to automatically attach that inheritance to the
override when its definition is encountered.

As the override might inherit from additional bases, this makes chasing
down "repeated base" errors a bit more involved.

Unfortunately, using this model, it must be illegal to instantiate or apply
sizeof() to a "virtual" class if an "override" is not visible in the same
translation unit.

[Chris Gary]

I would also like to add that, unless there is a reasonable way to combine
a "white" and "black" into the same definition, this actually *forces* the
two to be separated.

[Matthew Woehlke]

Post by Chris Gary
To avoid introducing new CSK, you can re-use "virtual" and "override" for
this purpose.

Hmm... not loving it. Anything wrong with:

public class Foo /* white */ { ... };
private class Foo /* black */ { ... };

...?

Post by Chris Gary
I would also like to add that, unless there is a reasonable way to combine
a "white" and "black" into the same definition, this actually *forces* the
two to be separated.

How would such a combination be different from a regular class?
--
Matthew

[mobi phil]

Post by Matthew Woehlke

Post by Chris Gary
To avoid introducing new CSK, you can re-use "virtual" and "override" for
this purpose.

public class Foo /* white */ { ... };
private class Foo /* black */ { ... };
...?

Post by Chris Gary
I would also like to add that, unless there is a reasonable way to

combine

Post by Chris Gary
a "white" and "black" into the same definition, this actually *forces*

the

Post by Chris Gary
two to be separated.

How would such a combination be different from a regular class?

sorry.. did you read the original post and the thread?

[mobi phil]

Post by mobi phil

Post by Chris Gary
To avoid introducing new CSK, you can re-use "virtual" and "override" for

Post by Chris Gary
this purpose.

public class Foo /* white */ { ... };
private class Foo /* black */ { ... };
...?

Post by Chris Gary
I would also like to add that, unless there is a reasonable way to

combine

Post by Chris Gary
a "white" and "black" into the same definition, this actually *forces*

the

Post by Chris Gary
two to be separated.

How would such a combination be different from a regular class?

sorry.. did you read the original post and the thread?

probably not bad idea for keywoard.. though

class public Foo {

}

and

class private Foo {

}

the "public class Foo" would rather mean that the class is public, whereas
it is about the part that is being declared is public/private.

[Matthew Woehlke]

Post by mobi phil

Post by Matthew Woehlke

Post by Chris Gary
I would also like to add that, unless there is a reasonable way
to combine a "white" and "black" into the same definition, this
actually *forces* the two to be separated.

How would such a combination be different from a regular class?

sorry.. did you read the original post and the thread?

It's been a while, but I think I recall the general gist. I think one of
us is missing something, however. I get the black/white concept when you
separate the interface from the implementation (isn't that the point?).
What I *don't* get is what it means to use this feature when the two are
*not* separated. (In particular, as per my question, what would be the
use case for that and how would it differ from a "regular" class?)
--
Matthew

[mobi phil]

Post by Matthew Woehlke
It's been a while, but I think I recall the general gist. I think one of
us is missing something, however. I get the black/white concept when you
separate the interface from the implementation (isn't that the point?).

exactly

Post by Matthew Woehlke
What I *don't* get is what it means to use this feature when the two are
*not* separated.

sorry... really confused. If you do not separate the the feature is not
present. They are hand in hand.. or I miss what you miss ;)

(In particular, as per my question, what would be the

Post by Matthew Woehlke
use case for that and how would it differ from a "regular" class?)

Take it like that: they don't want to differ from a regular classe. They
want to solve few problems:

1. keep it simple (the interface)
2. avoid useless dependency between interface consumer and the interface,
thus reduce recompilation in development process when you change layout of
the class that is not affecting consumers

2. depending on the implementation, other "positive side effects" like
loading classes from shared library, dynamic patching of the source code
etc.

[Chris Gary]

Post by Chris Gary
*snip*

Post by Chris Gary
I would also like to add that, unless there is a reasonable way to

combine

Post by Chris Gary
a "white" and "black" into the same definition, this actually *forces*

the

Post by Chris Gary
two to be separated.

How would such a combination be different from a regular class?
--
Matthew

The idea would be that it would simultaneously define an interface and
implementation, while allowing other implementations to be defined later.
This would reduce repetition, as a separate "black" would not have to be
defined right away.

Unless there is some way to add a "pure" specifier, the absence of which
would allow a "white" definition to double as "black".

I still have a problem with needing to resolve sizeof() at link time: Using
sizeof() is critical for templates that decide things like whether or not
an instance will fit in a small aligned buffer versus needing heap
allocation.

Obviously, alignment becomes a problem, as we might have some member with
larger than alignof(double) requirement (e.g. AVX tuple). I don't think
this can be dealt with efficiently at link time, as it would basically
require all template "algorithms" to be recorded as bytecode somehow. That,
or simply forbidding sizeof() or alignof() to be used on an interface in
the context of a template, which makes the idea incompatible with half the
language...

Concerning keywords: I don't really care what is chosen, just as long as
they don't turn nested definitions and the like into keyword soup or become
easily confused with other commonly-defined symbols (such as color enums).
My policy about keywords is that the result needs to be reasonably readable
as monochrome text...

[Chris Gary]

That, or simply forbidding sizeof() or alignof() to be used on an
interface in the context of a template, which makes the idea incompatible
with half the language...

I exaggerated a bit there... Let me clarify: If a "white" can double as a
"black" (or there is a compatible "black" in the TU), then sizeof() and
alignof() are well-defined, but it absolutely must be deduced at compile
time. The case where it can't be used in a template really isn't that
different than applying sizeof() to an incomplete type (pure "white" is
fundamentally incomplete)...

[mobi phil]

Post by Chris Gary
I exaggerated a bit there... Let me clarify: If a "white" can double as a
"black" (or there is a compatible "black" in the TU), then sizeof() and
alignof() are well-defined, but it absolutely must be deduced at compile
time.

what do you mean compatible? Same layout, same size? Well then we missed
the point of the full story. Again, I emphasize the reason of having chosen
the terms black and white. Black box is black box... thus it contains the

Post by Chris Gary
The case where it can't be used in a template really isn't that different
than applying sizeof() to an incomplete type (pure "white" is fundamentally
incomplete)...

still needs to be analyzed.. but I tend to think the same. If alloca is
standard then class could be allocated on stack. I still fail to see how
call by value could be solved if sizeof is not known.

Not sure if I mentioned in my original post, but this black/white story is
mainly intended for "big classes" which will not be candidate for call by
value.

[Chris Gary]

Post by mobi phil
still needs to be analyzed.. but I tend to think the same. If alloca is
standard then class could be allocated on stack. I still fail to see how
call by value could be solved if sizeof is not known.

The compiler has to reduce a 'sizeof(button)' or 'alignof(button)' to a
constant integer and fold any dependent constant expressions or templates
dependent thereupon before sending the whole thing to code generation. It
is not possible to know size or alignment if the precise content and layout
of a 'button' is only speculative or unspecified.

Not sure if I mentioned in my original post, but this black/white story is

Post by mobi phil
mainly intended for "big classes" which will not be candidate for call by
value.

Classes are treated as general categories, regardless of size. This
proposal will need to work for classes of any size, even empty ones.

[mobi phil]

Post by mobi phil
still needs to be analyzed.. but I tend to think the same. If alloca is

Post by mobi phil
standard then class could be allocated on stack. I still fail to see how
call by value could be solved if sizeof is not known.

The compiler has to reduce a 'sizeof(button)' or 'alignof(button)' to a
constant integer and fold any dependent constant expressions or templates
dependent thereupon before sending the whole thing to code generation. It
is not possible to know size or alignment if the precise content and layout
of a 'button' is only speculative or unspecified.

I did not wake up on this idea few days ago. Had it in my mind for a while
and it was clear for me from the beginning the two main issues: sizeof()
and layout (offsets). This two problems were repeated through the thread,
but it is my fault that I did not write a more formal paper where these
things are described so nobody wastes time to re-iterate the same thing. In
current implementation indeed not easy to fill in this gap, but IMHO is
doable.

Not sure if I mentioned in my original post, but this black/white story is

Post by mobi phil

Post by mobi phil
mainly intended for "big classes" which will not be candidate for call by
value.

Classes are treated as general categories, regardless of size. This
proposal will need to work for classes of any size, even empty ones.

The point I tried to make is that not knowing sizeof() at compilation time
may may cause some implementation issues for for call by value or
reference. If it turns out that will be impossible to implement call by
value or by reference, it should not be considered as a catastrophe and the
proposal rejected for this reason. But this may be just a false alarm due
to my limited knowledge of how call by reference and by value is
translated. Once I have time will have a look on how code is generated for
different layouts etc.

[Chris Gary]

Post by Chris Gary
*snip*

The point I tried to make is that not knowing sizeof() at compilation time

Post by Chris Gary
may may cause some implementation issues for for call by value or
reference. If it turns out that will be impossible to implement call by
value or by reference, it should not be considered as a catastrophe and the
proposal rejected for this reason. But this may be just a false alarm due
to my limited knowledge of how call by reference and by value is
translated. Once I have time will have a look on how code is generated for
different layouts etc.

This is why I keep repeating "sizeof() must reduce to a constant at compile
time." As templates constitute what is basically a functional language of
categories and simple integer expressions, we would need a way to modify
already-compiled binaries to accept a type with an unknown layout and
alignment (want some more bytecode with your bytecode?). I believe similar
issues were encountered with "export templates"... Imagine some exported
dependent template that does not terminate for some odd size of a type; how
difficult it might be to isolate that kind of bug (unless we have a
solution to the halting problem somewhere). So, keeping things simple, we
either forbid sizeof(), new, etc.. without a visible PIMPL, or just invoke
WYSIWYG.

[mobi phil]

Post by Chris Gary
*snip*
The point I tried to make is that not knowing sizeof() at compilation time

Post by mobi phil
may may cause some implementation issues for for call by value or
reference. If it turns out that will be impossible to implement call by
value or by reference, it should not be considered as a catastrophe and the
proposal rejected for this reason. But this may be just a false alarm due
to my limited knowledge of how call by reference and by value is
translated. Once I have time will have a look on how code is generated for
different layouts etc.

This is why I keep repeating "sizeof() must reduce to a constant at
compile time." As templates constitute what is basically a functional
language of categories and simple integer expressions, we would need a way
to modify already-compiled binaries to accept a type with an unknown layout
and alignment (want some more bytecode with your bytecode?). I believe
similar issues were encountered with "export templates"... Imagine some
exported dependent template that does not terminate for some odd size of a
type; how difficult it might be to isolate that kind of bug (unless we have
a solution to the halting problem somewhere). So, keeping things simple, we
either forbid sizeof(), new, etc.. without a visible PIMPL, or just invoke
WYSIWYG.

can we please someway agree that if sizeof has to reduce to a constant at
compile time is in serious contradiction what this proposal try to achive?

The point is that you add minimum to the header, you freeze it, and then
you do your development cycles in the cpp... Once the data layout is in the
black, you cannot have the sizeof.

About your argument above, do not say no, but one has to analyze how much
size is affecting templates and how much templates can do with the data
that stays in the white. Speculations in one direction or other do not
really help to much.

[Chris Gary]

This should illustrate my argument a little more clearly:

namespace _impl{

using AnySmallBfr = uintptr_t[3];

struct _basic_any_blob
{
virtual void _destroy() noexcept = 0;

// Destructor not called!
};

template<
typename InnerType
bool isSmall = (
sizeof(InnerType) <= sizeof(AnySmallBfr)
)
struct _any_blob : _basic_any_blob
{
InnerType _that;

void _destroy() noexcept override
{this->_that->~InnerType();}

// Rest of implementation
};

template<typename InnerType>
struct _any_blob<InnerType, false>
{
// Too big, need to allocate separately
InnerType *_that_ptr;

void _destroy() noexcept override
{delete this->_that_ptr;}

// Rest of implementation
};

}// namespace _impl

class any
{
_impl::AnySmallBfr _small_bfr;

public:

template<typename ValueType>
any(ValueType &&value)
{
// Decides between small and large
// backing store.
//
// If sizeof(ValueType) <= sizeof(_small_bfr),
// placement-new into _small_bfr.

// This should compile statically, but...
//
// Are you suggesting we automate code generation
// to handle structures like _any_blob<> parameterized
// by "white" types?
//
new(this->_small_bfr) _impl::_any_blob<
decay<ValueType>

{forward<ValueType>(value)};

}

~any() noexcept
{
reinterpret_cast<_impl::_basic_any_blob &>(
this->_small_bfr
)._destroy();
}
};

Consumer using an 'any':

class public thing
{
public:

string in_thing;
};

void verb()
{
// What happens if the resulting object's
// size depends on something loaded externally?
//
// Also: What if we have multiple implementations linked? Which 'thing'
is chosen?
//
any wut = thing{};
}

Producer with another thing:

// Too big for internal buffer of 'any'
//
class private thing
{
public:

string in_thing;
int hidden_int;
float hidden_float;
char hidden_chars[16];
};

[Chris Gary]

I need to point out the sample I gave was hacked together in a few minutes
and is riddled with bugs. The point here is illustrated in the constructor
of 'any'.

[mobi phil]

sorry, first before I try to understand the point you try to make with the
example, let's clarify one thing: you are intentionally using sizeof
(probably to explain your concern), whereas it is clear that what I am
proposing can be implemented if the dependency on the sizeof is eliminated
or can be delayed to link time or load time.

again: not all classes can be used in all situations. A class can be used
in a certain situation (template parameter etc.) if it implements a certain
protocol. In our case the class does not implement the compile time sizeof
protocol, thus it cannot be used in situation where this protocol is
needed. This does not mean that the proposal has no value etc. We are
playing with concept not with a universal machine, or theory of everything
:)

Post by Chris Gary
namespace _impl{
using AnySmallBfr = uintptr_t[3];
struct _basic_any_blob
{
virtual void _destroy() noexcept = 0;
// Destructor not called!
};
template<
typename InnerType
bool isSmall = (
sizeof(InnerType) <= sizeof(AnySmallBfr)
)
struct _any_blob : _basic_any_blob
{
InnerType _that;
void _destroy() noexcept override
{this->_that->~InnerType();}
// Rest of implementation
};
template<typename InnerType>
struct _any_blob<InnerType, false>
{
// Too big, need to allocate separately
InnerType *_that_ptr;
void _destroy() noexcept override
{delete this->_that_ptr;}
// Rest of implementation
};
}// namespace _impl
class any
{
_impl::AnySmallBfr _small_bfr;
template<typename ValueType>
any(ValueType &&value)
{
// Decides between small and large
// backing store.
//
// If sizeof(ValueType) <= sizeof(_small_bfr),
// placement-new into _small_bfr.
// This should compile statically, but...
//
// Are you suggesting we automate code generation
// to handle structures like _any_blob<> parameterized
// by "white" types?
//
new(this->_small_bfr) _impl::_any_blob<
decay<ValueType>

{forward<ValueType>(value)};

}
~any() noexcept
{
reinterpret_cast<_impl::_basic_any_blob &>(
this->_small_bfr
)._destroy();
}
};
class public thing
{
string in_thing;
};
void verb()
{
// What happens if the resulting object's
// size depends on something loaded externally?
//
// Also: What if we have multiple implementations linked? Which
'thing' is chosen?
//
any wut = thing{};
}
// Too big for internal buffer of 'any'
//
class private thing
{
string in_thing;
int hidden_int;
float hidden_float;
char hidden_chars[16];
};
--
---
You received this message because you are subscribed to the Google Groups
"ISO C++ Standard - Future Proposals" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at
http://groups.google.com/a/isocpp.org/group/std-proposals/.

--
rgrds,
mobi phil

being mobile, but including technology
http://mobiphil.com

[Thiago Macieira]

Post by mobi phil
The point I tried to make is that not knowing sizeof() at compilation time
may may cause some implementation issues for for call by value or
reference. If it turns out that will be impossible to implement call by
value or by reference, it should not be considered as a catastrophe and the
proposal rejected for this reason. But this may be just a false alarm due
to my limited knowledge of how call by reference and by value is
translated. Once I have time will have a look on how code is generated for
different layouts etc.

I can tell you right now: without knowing the size, passing by value in the
current way "pass-by-value" is understood will be impossible, period.

Passing by reference or by pointer should be possible, since those already
work even on incomplete types.

A solution for the by-value case is to treat it as passing by
pointer/reference in the ABI. That is, in most ABIs, the compiler will have to
reserve stack space for the dynamic size, copy the content, then pass the
address of that space.
--
Thiago Macieira - thiago (AT) macieira.info - thiago (AT) kde.org
Software Architect - Intel Open Source Technology Center
PGP/GPG: 0x6EF45358; fingerprint:
E067 918B B660 DBD1 105C 966C 33F5 F005 6EF4 5358

[mobi phil]

Post by Thiago Macieira
I can tell you right now: without knowing the size, passing by value in the
current way "pass-by-value" is understood will be impossible, period.
Passing by reference or by pointer should be possible, since those already
work even on incomplete types.
A solution for the by-value case is to treat it as passing by
pointer/reference in the ABI. That is, in most ABIs, the compiler will have to
reserve stack space for the dynamic size, copy the content, then pass the
address of that space.

that was exactly my concern, however, without any intention to offend: do
you exactly know the implementation difference between call by reference
and call by value? I am afraid from the called function point of view that
is NO difference. On the caller side you will have the original object and
a copy of it will be created before the call. Now if the original object
can be created with alloca then the copy also can be created with alloca.
Do not see any difference here.

[Thiago Macieira]

Post by mobi phil
that was exactly my concern, however, without any intention to offend: do
you exactly know the implementation difference between call by reference
and call by value?

Yes.

Under certain constraints, even aggregates might be passed in registers. This
really depends on the ABI and the psABI in question, so YMMV quite a lot. You
can read more about this in my blog:
http://www.macieira.org/blog/2012/02/the-value-of-passing-by-value/

Any type that can't be passed in registers is passed by hidden reference: a
copy is spilled to the stack and its address is passed in place of the type
itself.

Black or white types will always need to be passed by hidden reference.

Post by mobi phil
I am afraid from the called function point of view that
is NO difference. On the caller side you will have the original object and
a copy of it will be created before the call. Now if the original object
can be created with alloca then the copy also can be created with alloca.
Do not see any difference here.

--
Thiago Macieira - thiago (AT) macieira.info - thiago (AT) kde.org
Software Architect - Intel Open Source Technology Center
PGP/GPG: 0x6EF45358; fingerprint:
E067 918B B660 DBD1 105C 966C 33F5 F005 6EF4 5358

[mobi phil]

Post by Thiago Macieira
Black or white types will always need to be passed by hidden reference.

missing some optimization opportunities should not be interpreted as the
end of the world or as to say "period" like you stated earlier

Post by Thiago Macieira

Post by Thiago Macieira
I can tell you right now: without knowing the size, passing by value in the
current way "pass-by-value" is understood will be impossible, period.

can you please tell me, why you said it would be impossible? Now you seem
to have reduced the "impossible" to the handicap of not being able to pass
in registers. I would love to have time to read your blog and other
millions of books, article on the topic. Best is to look at generated code,
no blog/book/video/etc. can tell me more than. What I see so far that it is
feasable. If some mini optimization opportunities are lost, well, you
always have to tread sthg. for sthg. else...

I am personally ok with this little "disadvantage". If I want to hunt for
that little piece of cake, then I will move my blacks to the white and get
it once the project reached a stable phase. I am also thinking about some
details on how to move black back if optimization is to be achieved.

Unfortunately most of the projects I worked on there was less worry about
such optimization opportunities like passing an object in registers rather
than on stack. Happened often that this brilliant managers did read in some
fancy local newspapers that was mad to tell "-O{1,2,3}" to the compiler. It
is also true that 15 years ago optimizer were often producing buggy code.

Do not forget that QT pimple object cannot be passed in registers...
Because he is still a pointer...

So, lets take things easy. Let's do not adventure to far, before we did not
fix simple things. Let's do not optimize before we agreed on basic stuff...
Would alloca knowing the size from a function or global variable solve the
problem of sizeof of black?

[Thiago Macieira]

Post by mobi phil

Post by Thiago Macieira
Black or white types will always need to be passed by hidden reference.

missing some optimization opportunities should not be interpreted as the
end of the world or as to say "period" like you stated earlier

I never meant that it was the end of the world. I wrote quite explicitly that
it could never be done as it is currently done, that's it.

Post by mobi phil

Post by Thiago Macieira

Post by Thiago Macieira
I can tell you right now: without knowing the size, passing by value in

the

Post by Thiago Macieira

Post by Thiago Macieira
current way "pass-by-value" is understood will be impossible, period.

can you please tell me, why you said it would be impossible? Now you seem

How many registers do you reserve for the structure if you don't know its
size? Selecting the registers dynamically means all other parameters would be
dynamically determined, which seems to me to be horribly inefficient.

It's more efficient to allocate stack space, pass by hidden reference and keep
all the other parameters at fixed positions.

Post by mobi phil
to have reduced the "impossible" to the handicap of not being able to pass
in registers. I would love to have time to read your blog and other
millions of books, article on the topic.

Well, if you're not going to read up on the subject, how can we discuss the
subject?

Post by mobi phil
Best is to look at generated code,
no blog/book/video/etc. can tell me more than. What I see so far that it is
feasable. If some mini optimization opportunities are lost, well, you
always have to tread sthg. for sthg. else...

How do you think I came up with the content of the blog? By compiling and
generating code, plus reading the ABI manuals.

Post by mobi phil
I am personally ok with this little "disadvantage". If I want to hunt for
that little piece of cake, then I will move my blacks to the white and get
it once the project reached a stable phase. I am also thinking about some
details on how to move black back if optimization is to be achieved.

Once a black box, always a black box: as you don't know the other side, you
must assume the other side doesn't have access to the white box.

Post by mobi phil
Do not forget that QT pimple object cannot be passed in registers...
Because he is still a pointer...

The Private class can't, but the outer, public class could be.

Though for Qt, classes that can be passed in registers are rare because most
are either ref-counted or non-copyable and neither of those types can be
passed in registers.

Post by mobi phil
So, lets take things easy. Let's do not adventure to far, before we did not
fix simple things. Let's do not optimize before we agreed on basic stuff...
Would alloca knowing the size from a function or global variable solve the
problem of sizeof of black?

Yes. if the compiler can do:

ClassName object = otherObject;

It can pass by value.
--
Thiago Macieira - thiago (AT) macieira.info - thiago (AT) kde.org
Software Architect - Intel Open Source Technology Center
PGP/GPG: 0x6EF45358; fingerprint:
E067 918B B660 DBD1 105C 966C 33F5 F005 6EF4 5358

[mobi phil]

Post by Thiago Macieira
How many registers do you reserve for the structure if you don't know its
size? Selecting the registers dynamically means all other parameters would be
dynamically determined, which seems to me to be horribly inefficient.

without pretending to be an assembler or code generation guru, I would say
one for the pointer and accept that no such optimisations are possible. If
such optimisations are desired in some cases, then don't use from the
beginnig black/white or move back to normal class at certain moment in the
development when the interface is stable.

Post by Thiago Macieira

Post by mobi phil
to have reduced the "impossible" to the handicap of not being able to

pass

Post by mobi phil
in registers. I would love to have time to read your blog and other
millions of books, article on the topic.

Well, if you're not going to read up on the subject, how can we discuss the
subject?

My curiosity is there, but not the time unfortunately. Though the devil
often is hidden in details, I would like to know first at "high level" what
are the pitfalls and what are the possibilities. But at this stage we
clarified that both passing by value/reference and pointer would be
possible without knowing the size at compilation time.

If we agree that code-generation-time optimization decision like passing
through registrer depends on sizeof and sizeof is not available, do not see
the point for me to spend lot of time to understand the details.

Post by Thiago Macieira

Post by mobi phil
Best is to look at generated code,
no blog/book/video/etc. can tell me more than. What I see so far that it

is

Post by mobi phil
feasable. If some mini optimization opportunities are lost, well, you
always have to tread sthg. for sthg. else...

How do you think I came up with the content of the blog? By compiling and
generating code, plus reading the ABI manuals.

great. Reading of your post is on my todo. Right now I am busy to finish
this mini project for gdb interface.

Post by Thiago Macieira
I am personally ok with this little "disadvantage". If I want to hunt for

Post by mobi phil
that little piece of cake, then I will move my blacks to the white and

get

Post by mobi phil
it once the project reached a stable phase. I am also thinking about some
details on how to move black back if optimization is to be achieved.

Once a black box, always a black box: as you don't know the other side, you
must assume the other side doesn't have access to the white box.

Sorry... did not understand this as a whole, but pb. not important

Post by Thiago Macieira

Post by mobi phil
Do not forget that QT pimple object cannot be passed in registers...
Because he is still a pointer...

The Private class can't, but the outer, public class could be.

Was talking indeed about the private. So far probably there were no serious
complaints about it. And qt is nice and fast.

Post by Thiago Macieira
Though for Qt, classes that can be passed in registers are rare because most
are either ref-counted or non-copyable and neither of those types can be
passed in registers.

So, I am happy that we both come to the came conclusion. So I should not
worry too much that missing the registers would be a problem in case of
blacks.

Post by Thiago Macieira

Post by mobi phil
So, lets take things easy. Let's do not adventure to far, before we did

not

Post by mobi phil
fix simple things. Let's do not optimize before we agreed on basic

stuff...

Post by mobi phil
Would alloca knowing the size from a function or global variable solve

the

Post by mobi phil
problem of sizeof of black?

ClassName object = otherObject;
It can pass by value.

Well, then this topic is rather clear.

What I still need to work out is how private data member access (inside
black) will be expressed (syntax) and implemented. The topic was touched in
earlier emails. It was proposed to use public/private keywords for
black/white and eventually the keyword virtual, for members in white that
are "implemented' in he black. The question is either one should go with :

-> functions or
-> pointers

Manually written or automatically generated (also to be decided) functions
could be inlined at link time (llvm lto, hopefully this technique will
catch-up). This would generate faster code but has the disadvantage that
will not be able to load classes from shared libraries which would be
another advantage of this proposal. Probably both implementations would
make sense, the question is how to make the distinction.

[mobi phil]

Post by Chris Gary
The idea would be that it would simultaneously define an interface and
implementation, while allowing other implementations to be defined later.
This would reduce repetition, as a separate "black" would not have to be
defined right away.

Did not understand this entirely. One of the main objective is to keep
class interfaces thus "whites" untouched during development cycles. Thus
cannot see too much reason not to have separate black. can you give an
example pls.

Post by Chris Gary
Unless there is some way to add a "pure" specifier, the absence of which
would allow a "white" definition to double as "black".

again, I fail to see the point, but please develop, give example

Post by Chris Gary
Using sizeof() is critical for templates that decide things like whether or
not an instance will fit in a small aligned buffer versus needing heap
allocation.

Well, for sure it is not easy. Maybe such templates will be forbidden to
use such constructs, unless adapted.

Post by Chris Gary
Obviously, alignment becomes a problem, as we might have some member with
larger than alignof(double) requirement (e.g. AVX tuple). I don't think
this can be dealt with efficiently at link time, as it would basically
require all template "algorithms" to be recorded as bytecode somehow. That,
or simply forbidding sizeof() or alignof() to be used on an interface in
the context of a template, which makes the idea incompatible with half the
language...

Might be an issue, but do not understand it. I do not think allignment
would be an issue in places where it was not an issue, or it will be an
issue where it was an issue. In the class layout the alignment will be as
before, with same freedom and restrictions. Do you mean at runtime when the
members are referenced? Well... I need help there... did not study the
issue.

Post by Chris Gary
Concerning keywords: I don't really care what is chosen, just as long as
they don't turn nested definitions and the like into keyword soup or become
easily confused with other commonly-defined symbols (such as color enums).
My policy about keywords is that the result needs to be reasonably readable
as monochrome text...

I am kind of convinced that so far the syntax should be

class private {... }
class public {...}

[Chris Gary]

*snip* Thus cannot see too much reason not to have separate black. can you
give an example pls.

Consumer:

class public button
{
public:

string text;
drawing::color color;
};

[[dllimport]]
button *make_special_button();

void verb()
{
// WYSIWYG instantiation of button.
//
// sizeof(button) MUST reduce to a constant integer at compile time!
//
button btn{};

button *btn_ptr = make_special_button();

btn_ptr->text = "Press me!";
}

PIMPL producer:

// Problem:
//
// What if more than one of these are visible in the same TU?
// Use the most recent definition?
// If they all have the same layout and member ordering, don't complain?
//
class private button
{
public:

// To save typing, this automatically injects
// members that aren't explicitly declared
// in the same declaration order, before any
// member declared in "private"

// "text" implicitly declared before what follows.

HWND hWnd;
drawing::color color;
};

[[dllexport]]
button *make_special_button()
{
return new button{/*Create window, set text, blah...*/};
}

[mobi phil]

try to find also my comments in your example

*snip* Thus cannot see too much reason not to have separate black. can you

give an example pls.

class public button
{
string text; // MOBIPHIL: still needs to be decided if this means a
"real" data that extends the interface or automatically a "virtual" one
that needs to be resolved by a black
drawing::color color;
};
[[dllimport]]
button *make_special_button();
void verb()
{
// WYSIWYG instantiation of button.
//
// sizeof(button) MUST reduce to a constant integer at compile time!
MOBIPHIL: not necessarily! if behind the scene the compiler could generate
sthg. like char *buffer = alloca(button_sizeof()); new (buffer) blahblah...
//
button btn{};
button *btn_ptr = make_special_button();
btn_ptr->text = "Press me!";
}
//
// What if more than one of these are visible in the same TU?
// Use the most recent definition? MOBIPHIL: you will get duplicated
symbol for symbols (either functions or globals, depending on the
implementation) that are defining the sizeof and offsets of visible members
// If they all have the same layout and member ordering, don't complain?
//
class private button
{
// To save typing, this automatically injects
// members that aren't explicitly declared
// in the same declaration order, before any
// member declared in "private" MOBIPHIL: no real objections against
injection... see further comment below
// "text" implicitly declared before what follows.
HWND hWnd;
drawing::color color;
};
[[dllexport]]
button *make_special_button()
{
return new button{/*Create window, set text, blah...*/};
}

for me the black will define some symbols that need to be available at link
time, such symbols are black_sizeof(), and offsets of the attributes that
are accessed from consumers either as setter/getter or through this
"virtual member" mechanism that makes sense. Any "white" declaration will
create dependency on such symbols, thus if the black is not available then
these symbols would be not defined. Remember that I am still proposing two
approaches for symbol resolution
-> link time (link time inlining of setters/getters ... thus the virtual
data members)
-> load time with pointers

The injection you mention in your example can make sense but there are two
contradictory approaches, I was riding more the second:

1. what you mention here that the memebres get automatically injected from
the "public" (white) and a link should be made someway between the two.
This will not allow however the option to have "real" public members.
Nevertheless allowing "real" public members would make implementation even
more difficult.

2. The other way is to have a distinction between "virtual" and normal
members on the "white". Virtual ones would be linked with the one on the
black.

In the design I am envisaging you do not need the factory method
make_special_button. However it will work like a factory method.

[mobi phil]

hi, as stated in the original post, personally do not care how the two
constructs will be named, thus if majority of folks would love
virtual/override do not care... However it is not really virtual and
override in my original concept. I would not force all the data members to
be virtual. I would allow part of data to be in white. The black would
complement it. This in my concept the title from white will be different
than the title from black, well, there should be no duplicates however.
What is also open for "vote" how one would link the data member from black
to the interface. One way would be sthg. you are mentioning, the other
through manually specified getter/setters.

Post by Chris Gary
class white button
{
// Speculative data members
// (roughly equivalent to properties)
virtual string title; /* note I added here virtual */
// // it would be equally possible string getTitle();
/* note I added here virtual */
visual_style &style;
};
class black button
{
string title; /* the linker will make this public through the
virtual from white */
HWND hWnd;
visual_style &style;
// Etc...
};
Accessing a member through a speculative interface would merely involve
fetching an offset from a vtable. Much like static binding of a virtual
method call, it is also possible to deduce a member offset statically if
the concrete definition is available.

indeed, if you read through the thread vtable and other "synonims" were
mentioned. I used global offset table, but all is about accessing an offset
from a location that is known only at link time or load time versus the
"efficient" hard coded way.


Inheriting from a virtual class makes the result virtual and requires that

Post by Chris Gary
the override inherit from the same bases (possibly in the same order). It
might be reasonable to automatically attach that inheritance to the
override when its definition is encountered.

well, I am afraid this would make things rather complicated and lower the
chance for acceptance. In the original concept the inheritance and lot of
things would be like in the normal classes to avoid complications. One
could translate the original proposal into: make invisible what is indeed
private, but keep the rest of the rules unchanged

Post by Chris Gary
As the override might inherit from additional bases, this makes chasing
down "repeated base" errors a bit more involved.

Hm... again... seems for me unnecessary complication unless you come with
example and use case.

Post by Chris Gary
Unfortunately, using this model, it must be illegal to instantiate or
apply sizeof() to a "virtual" class if an "override" is not visible in the
same translation unit.

Indeed, this was mentioned in the original post if I am not wrong, and was
debated. For dynamic allocation the allocator should call a sizeof() member
function or will be a global variable calculated at link time.


--

Post by Chris Gary
---
You received this message because you are subscribed to the Google Groups
"ISO C++ Standard - Future Proposals" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at
http://groups.google.com/a/isocpp.org/group/std-proposals/.

--
rgrds,
mobi phil

being mobile, but including technology
http://mobiphil.com

[Matthew Woehlke]

Post by mobi phil
hi, as stated in the original post, personally do not care how the two
constructs will be named, thus if majority of folks would love
virtual/override do not care... However it is not really virtual and
override in my original concept. I would not force all the data members to
be virtual. I would allow part of data to be in white. The black would
complement it.

Agreed, but note also I don't think we're suggesting that. (Overriding
(pardon the pun) those keywords for this is a bit abusive of what they
normally mean, and a source of potential confusion as you've just
demonstrated, which is one reason using public/private instead might be
better.)
--
Matthew

[i***@gmail.com]

I'm sorry to involve in this way, but could you please state what is/are
the goals you want to achieve by this suggestion/proposal?
I haven't read all the posts under this topic, I'll just summarize the
goals I've noticed:
1) separate public interface from internal implementation details
2) enable change of internal implementation without the need of
recompilation code that only uses the public interface of the class/struct.
Please could you add other goals which I've missed? I'm sorry if I
completely misunderstood your ideas.

An old way to achieve goals 1&2: abstract interface with pure virtual
methods:
// white.h:
class IWidget {
public:
virtual void set(int) = 0;
virtual int get() const = 0;
};

// black.h:
class WidgetImpl : public IWidget {
public:
virtual void set(int i) override { x = i; }
virtual int get() const { return x; }

private:
int x;
};
... now if you decide to change the implementation of WidgetImpl, there's
no need to recompile code that uses only IWidget.




I can't imagine setup where this same behavior would be achieved by using
non-virtual methods. Let's consider code using the black/white keywords (I
might missed the proposed syntax, please correct me if I'm wrong):
// white.h:
class white Widget {
public:
void set(int i) { setInternal(i); }
int get() const { return getInternal(); }
};

// black.h:
class black Widget {
private:
void setInternal(int i) { x = i; }
int getInternal() const { return getInternal2(); }

private:
int getInternal2() const { return getInternal3(); }
int getInternal3() const { return x; }

private:
int x;
};

Then let's assume a use of Widget:
// foo.cpp:
#include "white.h"
void foo() {
Widget w;
w.set(2);
std::cout << w.get() << endl;
}
... here I think any reasonable compiler would inline the call of
Widget::get to just reading directly the private member Widget::x (there is
no virtual call, no obstacles (if not the black/white keywords)).
But by inlining, the compiled object file of foo.cpp now holds information
about not only internal structure of black Widget but also about its
implementation. So whenever the implementation of black Widget changes,
foo.cpp has to be recompiled to refresh that information.
IMHO either:
a) foo can see the entrails of black Widget -> then foo.cpp must be
recompiled on every black Widget change. (we didn't achieve goal #2)
or:
b) foo can be optimized only up to the implementation of white Widget, not
further. -> then it seems to me similar to calling a virtual method. (Yes,
it might be more efficient then calling an explicitly virtual method. But
I'm not sure if all this brings enough value).

[mobi phil]

I'm sorry to involve in this way, but could you please state what is/are the
goals you want to achieve by this suggestion/proposal?
I haven't read all the posts under this topic, I'll just summarize the goals
1) separate public interface from internal implementation details
2) enable change of internal implementation without the need of
recompilation code that only uses the public interface of the class/struct.
Please could you add other goals which I've missed? I'm sorry if I
completely misunderstood your ideas.
An old way to achieve goals 1&2: abstract interface with pure virtual
class IWidget {
virtual void set(int) = 0;
virtual int get() const = 0;
};
class WidgetImpl : public IWidget {
virtual void set(int i) override { x = i; }
virtual int get() const { return x; }
int x;
};
... now if you decide to change the implementation of WidgetImpl, there's no
need to recompile code that uses only IWidget.

anybody is welcome to be involved. Nevertheless your example just
introduces a level the dependency but does note break it. The entity
that will create WidgetImpl will still be dependent on the layout of
the class and you are introducing a parallel hierarchy exactly like
PIMPL. Plus dependency wise this is worst IMHO that PIMLP as the the
code that creates WidgetImpl still depends on WidgetImpl, or you
introduce a third hierarchy of factories.

I can't imagine setup where this same behavior would be achieved by using non-virtual methods.

Yes, PIMPL does sthg. similar, but not with virtuals but with
delegation. The proposal wants to simplify all this.

Let's consider code using the black/white keywords (I

class white Widget {
void set(int i) { setInternal(i); }
int get() const { return getInternal(); }
};
class black Widget {
void setInternal(int i) { x = i; }
int getInternal() const { return getInternal2(); }
int getInternal2() const { return getInternal3(); }
int getInternal3() const { return x; }
int x;
};
#include "white.h"
void foo() {
Widget w;
w.set(2);
std::cout << w.get() << endl;
}
... here I think any reasonable compiler would inline the call of
Widget::get to just reading directly the private member Widget::x (there is
no virtual call, no obstacles (if not the black/white keywords)).

Did you check yourself that, or it is pure assumption. Such
assumptions are making discussions more difficult. You are practically
delegating the work to the other side to prove the opposite. The
proposal will contains reccomandation to do such inlining at link time
which in my opinion.


Sorry, I am afraid that the compiler cannot inline something it is not
aware about. When you compile a unit that includes white.h (above) is
not aware about black.h, unless you explicitly include it, which does
not result from your example! How does your foo.cpp know about
black.h? Just magically??


Please verify first your assumptions and come back with a cleaner example.

regards,
mobiphil

[i***@gmail.com]

You're right, I must apologize for my confusion in what compiler and

linker does. The inlining is done in linker, not in compiler, so my
assumption about foo.cpp object file is completely wrong.

As I look at it, this proposal can cause a lot of work to be moved from
compiler to linker. I think it would be interesting to do some benchmarking
if a complete rebuild (compilation and linking) after an implementation
change is much faster then in current design.
If we move a lot of work out of compilation, then it's quite prospective,
that the compilation itself will be faster. -> there's a less difference if
the compilation actually happens or not.

[mobi phil]

You're right, I must apologize for my confusion in what compiler and linker
does. The inlining is done in linker, not in compiler, so my assumption
about foo.cpp object file is completely wrong.

As far as I know at the moment there is no inlining done in linker in
legacy compilation. Some new features of llvm lto allows link time
inlineing and the proposal among others will try to build on that
feature.

As I look at it, this proposal can cause a lot of work to be moved from
compiler to linker. I think it would be interesting to do some benchmarking
if a complete rebuild (compilation and linking) after an implementation
change is much faster then in current design.

Obviously no benchmarking will be possible till it is implemented.
Link time inlining is an option of the proposal, but I assume that the
link time inlining will be faster if it will not have to re-inline for
objects that did not change, so the build system itself will have an
impact on this.

[i***@gmail.com]

I think that is possible to benchmark 90% of this functionality today. Only
thing we lack is `sizeof` but this can be work around by using
`std::aligned_storage` that have size margin (you should be able add couple
of new members before you run out of space in it).


Another thing is that on C++ VLA committee reject runtime `sizeof`, do you
think they will allow it in your proposal? Even if we allow it you still
can't add members of that type to another class. Its because every access
of members declared after it will require function call for correct offset.
I think it would be better if we require that "black" declare max size of
"white" part. This will made it a lot of easer to implement (e.g. it will
be possible to do today with lot of boilerplate) and pass committee.

Post by mobiphil

You're right, I must apologize for my confusion in what compiler and

linker

does. The inlining is done in linker, not in compiler, so my assumption
about foo.cpp object file is completely wrong.

As far as I know at the moment there is no inlining done in linker in
legacy compilation. Some new features of llvm lto allows link time
inlineing and the proposal among others will try to build on that
feature.

As I look at it, this proposal can cause a lot of work to be moved from
compiler to linker. I think it would be interesting to do some

benchmarking

if a complete rebuild (compilation and linking) after an implementation
change is much faster then in current design.

Obviously no benchmarking will be possible till it is implemented.
Link time inlining is an option of the proposal, but I assume that the
link time inlining will be faster if it will not have to re-inline for
objects that did not change, so the build system itself will have an
impact on this.

[mobi phil]

Post by i***@gmail.com
I think that is possible to benchmark 90% of this functionality today. Only
thing we lack is `sizeof` but this can be work around by using
`std::aligned_storage` that have size margin (you should be able add couple
of new members before you run out of space in it).

sorry, how did you calculate this 90%? And bench marking what exactly?
The design is not yet even ready, we do not know what needs to be
changed in the compilers and linkers. Benchmarking which build? Can
you please make more clear statements?

Post by i***@gmail.com
Another thing is that on C++ VLA committee reject runtime `sizeof`, do you
think they will allow it in your proposal?

Did they reject in the past? Do they always reject? You assume they
are going to reject? Can you also pls. be more clear here?

Post by i***@gmail.com
Even if we allow it you still
can't add members of that type to another class. Its because every access of
members declared after it will require function call for correct offset.

Sorry, this one not clear either? You mean function call to get the
offset? If you have the function to get the offset, why would it be a
problem. You are contradicting yourself in the same statement. Another
option to offset function is the link time inlining. Please read
through the thread, you may have some of the questions clarified.

Post by i***@gmail.com
I think it would be better if we require that "black" declare max size of
"white" part. This will made it a lot of easer to implement (e.g. it will be
possible to do today with lot of boilerplate) and pass committee.

Max size is out of question. It is hack, I would personally downwoat
sthg. like that. How would you decide on max size? You will have 300
hundred classes and you will spend 30% of effort in adjusting the max
sizes? That would work in o toy application, but not on giant
projects. Would you just waste memory because design is bad and memory
is cheap?

[i***@gmail.com]

Post by i***@gmail.com

Post by i***@gmail.com
I think that is possible to benchmark 90% of this functionality today.

Only

Post by i***@gmail.com
thing we lack is `sizeof` but this can be work around by using
`std::aligned_storage` that have size margin (you should be able add

couple

Post by i***@gmail.com
of new members before you run out of space in it).

sorry, how did you calculate this 90%? And bench marking what exactly?
The design is not yet even ready, we do not know what needs to be
changed in the compilers and linkers. Benchmarking which build? Can
you please make more clear statements?
This is approximation, meaning of this was that is possible to recreate

most of this using current tools, but without any syntax support.

Post by i***@gmail.com

Post by i***@gmail.com
Another thing is that on C++ VLA committee reject runtime `sizeof`, do

you

Post by i***@gmail.com
think they will allow it in your proposal?

Did they reject in the past? Do they always reject? You assume they
are going to reject? Can you also pls. be more clear here?
All proposal that didn't have fixed size objects avoided declaring

`sizeof` for them. e.g.
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2014/n3875.pdf
And I recall some discussion about it, that people form committee didn't
like it.
I tired find link to it but right now I dint find it, maybe someone els
remember this?

Post by i***@gmail.com
Even if we allow it you still

Post by i***@gmail.com
can't add members of that type to another class. Its because every

access of

Post by i***@gmail.com
members declared after it will require function call for correct offset.

Sorry, this one not clear either? You mean function call to get the
offset? If you have the function to get the offset, why would it be a
problem. You are contradicting yourself in the same statement. Another
option to offset function is the link time inlining. Please read
through the thread, you may have some of the questions clarified.

class black A1 { void foo(); };
class black A2 { void foo(); };


class B
{
int i;
A1 a1;
int c;
A2 a2;
int d;
};

int main()
{
B b;
b.c = 5; //require function call to get offset value.
b.d = 3; //2x call for A1 and A2
b.a2.foo();
temp<&B::d> something; //error, this require runtime values.
int i = sizeof(b); //runtime value too
}

Post by i***@gmail.com
I think it would be better if we require that "black" declare max size of

Post by i***@gmail.com
"white" part. This will made it a lot of easer to implement (e.g. it

will be

Post by i***@gmail.com
possible to do today with lot of boilerplate) and pass committee.

Max size is out of question. It is hack, I would personally downwoat
sthg. like that. How would you decide on max size? You will have 300
hundred classes and you will spend 30% of effort in adjusting the max
sizes? That would work in o toy application, but not on giant
projects. Would you just waste memory because design is bad and memory
is cheap?

No, I'm more concern with complexity of this when another objects depends
on run time size of another object.

[mobi phil]

Post by mobi phil
sorry, how did you calculate this 90%? And bench marking what exactly?
The design is not yet even ready, we do not know what needs to be
changed in the compilers and linkers. Benchmarking which build? Can
you please make more clear statements?

This is approximation, meaning of this was that is possible to recreate most
of this using current tools, but without any syntax support.

Still not clear what you want to measure. Even if it would be already
implemented you would have hard time to measure. You measure what
against what? Build times would be clearly 1000 times faster if you do
not have to recompile 1000 object files with same complexity each time
you change the definition of a class they are depending on. The same
time you can have a simple build where this will bring zero added
value, more it will be a little bit slower. About runtime performance
difference: there will be to solutions. link time inlining or "runtime
sizeof". The first will be a bit slower to build, the second will be
slower to run. How much slower? dependends on the situation. If you
have 50 times access to the same member inside one function the
compiler is supposed to cache the offset. So the degradation will not
be probaly that significant, but again will depend on the situation.
If getters/setters will be inlined by the linker, I expect almost zero
degradation.

Please define what and how you want to measure.

Post by mobi phil

Post by i***@gmail.com
Another thing is that on C++ VLA committee reject runtime `sizeof`, do you
think they will allow it in your proposal?

Did they reject in the past? Do they always reject? You assume they
are going to reject? Can you also pls. be more clear here?

All proposal that didn't have fixed size objects avoided declaring `sizeof`
for them. e.g.
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2014/n3875.pdf
And I recall some discussion about it, that people form committee didn't
like it.
I tired find link to it but right now I dint find it, maybe someone els
remember this?

Please do not confuse with some proposals that had as object
dynamically growing objects or anything in that direction. In this
proposal the size is not known at compile time, but it will not change
at runtime. The proposal however keeps it open dynamically loaded
classes and in that case offset and size have to be dynamic.

Post by mobi phil

Post by i***@gmail.com
Even if we allow it you still
can't add members of that type to another class. Its because every access of
members declared after it will require function call for correct offset.

Sorry, this one not clear either? You mean function call to get the
offset? If you have the function to get the offset, why would it be a
problem. You are contradicting yourself in the same statement. Another
option to offset function is the link time inlining. Please read
through the thread, you may have some of the questions clarified.

class black A1 { void foo(); };
class black A2 { void foo(); };
class B
{
int i;
A1 a1;
int c;
A2 a2;
int d;
};
int main()
{
B b;
b.c = 5; //require function call to get offset value.
b.d = 3; //2x call for A1 and A2
b.a2.foo();
temp<&B::d> something; //error, this require runtime values.
int i = sizeof(b); //runtime value too
}

Did you think how you would implement sizeof at runtime? Without being
a compiler or assembler guru, I see three solutions (all these details
were described already in this thread):

1. having the linker going through all the generated code and replace
occurences of some "markers" with the sizeof available at link time
from the black
2. indirect reference. The sizeof would be an offset in a global table
that is arranged by the dynamic loader
3. with function()

The same applies to offsets

[i***@gmail.com]

Post by mobi phil

Post by i***@gmail.com

Post by mobi phil
sorry, how did you calculate this 90%? And bench marking what exactly?
The design is not yet even ready, we do not know what needs to be
changed in the compilers and linkers. Benchmarking which build? Can
you please make more clear statements?

This is approximation, meaning of this was that is possible to recreate

most

Post by i***@gmail.com
of this using current tools, but without any syntax support.

Still not clear what you want to measure. Even if it would be already
implemented you would have hard time to measure. You measure what
against what? Build times would be clearly 1000 times faster if you do
not have to recompile 1000 object files with same complexity each time
you change the definition of a class they are depending on. The same
time you can have a simple build where this will bring zero added
value, more it will be a little bit slower. About runtime performance
difference: there will be to solutions. link time inlining or "runtime
sizeof". The first will be a bit slower to build, the second will be
slower to run. How much slower? dependends on the situation. If you
have 50 times access to the same member inside one function the
compiler is supposed to cache the offset. So the degradation will not
be probaly that significant, but again will depend on the situation.
If getters/setters will be inlined by the linker, I expect almost zero
degradation.
Please define what and how you want to measure.

I see I miss your point, because I think of runtime benchmarking for the
whole time.

Post by mobi phil

Post by i***@gmail.com

Post by mobi phil

Post by i***@gmail.com
Another thing is that on C++ VLA committee reject runtime `sizeof`,

do

Post by i***@gmail.com

Post by mobi phil

Post by i***@gmail.com
you
think they will allow it in your proposal?

Did they reject in the past? Do they always reject? You assume they
are going to reject? Can you also pls. be more clear here?

All proposal that didn't have fixed size objects avoided declaring

`sizeof`

Post by i***@gmail.com
for them. e.g.
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2014/n3875.pdf
And I recall some discussion about it, that people form committee didn't
like it.
I tired find link to it but right now I dint find it, maybe someone els
remember this?

Please do not confuse with some proposals that had as object
dynamically growing objects or anything in that direction. In this
proposal the size is not known at compile time, but it will not change
at runtime. The proposal however keeps it open dynamically loaded
classes and in that case offset and size have to be dynamic.

But in both behavior of `sizeof` will change. Probably biggest problem will
be using it in metaprograming simple code like that: `test<(sizeof(x) >
4)>`.

Post by mobi phil

Post by i***@gmail.com

Post by mobi phil
Sorry, this one not clear either? You mean function call to get the
offset? If you have the function to get the offset, why would it be a
problem. You are contradicting yourself in the same statement. Another
option to offset function is the link time inlining. Please read
through the thread, you may have some of the questions clarified.

class black A1 { void foo(); };
class black A2 { void foo(); };
class B
{
int i;
A1 a1;
int c;
A2 a2;
int d;
};
int main()
{
B b;
b.c = 5; //require function call to get offset value.
b.d = 3; //2x call for A1 and A2
b.a2.foo();
temp<&B::d> something; //error, this require runtime values.
int i = sizeof(b); //runtime value too
}

Did you think how you would implement sizeof at runtime? Without being
a compiler or assembler guru, I see three solutions (all these details
1. having the linker going through all the generated code and replace
occurences of some "markers" with the sizeof available at link time
from the black
2. indirect reference. The sizeof would be an offset in a global table
that is arranged by the dynamic loader
3. with function()
The same applies to offsets

I think only first one is good for this. But "markers" will probably don't
work, because multiple black can affect one `sizeof`.
Probably more possible implementation would be list of instruction address
that need added value of black size to const operand.

Second have two problems, one that require constant keeping this table in
cache when you use black objects another is that all offset instruction
will require couple step to calculate final offset.

Third one will probably not pass because of overhead compared to accessing
meber by const offset. Its posible to store value, but it will be
recalculated for each new context.
And if it done properly it probably end up as second point.

Overall most of my concerns could be avoided by first implementation, but
template problem is unfixable.

[mobi phil]

Post by i***@gmail.com
I see I miss your point, because I think of runtime benchmarking for the
whole time.

Even if we talk about runtime bench-marking, the measurement would be
complex. You cannot measure without defining the problem space. If you
have suggestions, please define what you would like to measure. Just
throwing in numbers and random ideas does not help.

Post by i***@gmail.com
But in both behavior of `sizeof` will change. Probably biggest problem will
be using it in metaprograming simple code like that: `test<(sizeof(x) >
4)>`.

May I kindly ask you to go through all emails. This problem was
already addressed. The proposal does not intend to replace all the
usage of all the classes in all the templates in all the world. There
are classes that do not provide other kind of protocols that make them
unusable in thousands of other templates. The proposal will not
satisfy the interface for sizeof at compiltime and with that this part
of topic is closed.

Post by i***@gmail.com

Post by mobi phil
1. having the linker going through all the generated code and replace
occurences of some "markers" with the sizeof available at link time
from the black
2. indirect reference. The sizeof would be an offset in a global table
that is arranged by the dynamic loader
3. with function()
The same applies to offsets

I think only first one is good for this. But "markers" will probably don't
work, because multiple black can affect one `sizeof`.
Probably more possible implementation would be list of instruction address
that need added value of black size to const operand.
Second have two problems, one that require constant keeping this table in
cache when you use black objects another is that all offset instruction will
require couple step to calculate final offset.
Third one will probably not pass because of overhead compared to accessing
meber by const offset. Its posible to store value, but it will be
recalculated for each new context.
And if it done properly it probably end up as second point.
Overall most of my concerns could be avoided by first implementation, but
template problem is unfixable.

did you ever heard of the term trading something against something?
Did you hear about the theory of relativity? If you get something then
often you have to give up something else. So it is with this proposal!
It does not pretend to make the word much better. It wants to be a
solution for a certain problem. During development if one realises
that the black/white means serious performance, it should be possible
to make the black/white into an original class

[David Krauss]

Post by mobi phil
May I kindly ask you to go through all emails. This problem was
already addressed. The proposal does not intend to replace all the
usage of all the classes in all the templates in all the world. There
are classes that do not provide other kind of protocols that make them
unusable in thousands of other templates. The proposal will not
satisfy the interface for sizeof at compiltime and with that this part
of topic is closed.

You can summarily stop a part of this discussion so easily now, but once the committee realizes that you’re proposing sizeof expressions that aren’t compile-time constants, they’ll stop the entire proposal equally swiftly. I’ve not been following this thread, but you might be better off saying that sizeof cannot be applied to certain types at all, and introducing a new runtime function to take its place if necessary.

By the way, the VLA/“classes of runtime size” proposal did not provide for varying object size at runtime, to my knowledge.

[David Krauss]

Post by David Krauss
By the way, the VLA/“classes of runtime size” proposal did not provide for varying object size at runtime, to my knowledge.

I mean, changing the size of an object already created, as suggested a couple messages ago. Of course it allowed for deciding the size of an object upon construction.

[mobi phil]

Post by David Krauss
I mean, changing the size of an object already created, as suggested a
couple messages ago. Of course it allowed for deciding the size of an object
upon construction.

Jumping in and while landing grabbing a random apple from the tree,
would not bring too much added value, isn't it? ;)

Can you please read more carefully the thread and avoid statements
like "changing the size of an object already created, as suggested a
couple messages ago..". At least myself I did not make such
irresponsible statements. You confused with variations on the idea
about loading the full class from a dynamic library which means
together with sizeof() and equivalent functions and offset providing
functions that satisfy the white interface.

You make me repeating the same trivial statement yet the x'th time:
the runtime "sizeof" detail is an option. The main proposal is
focusing on link time inlining. Thus you missed the point, it wont be
compile time constant sizeof, it wont't be runtime, but it wants to be
link time constant. For the rest if you bother, just go through the
thread.

[David Krauss]

Post by mobi phil
Can you please read more carefully the thread and avoid statements
like "changing the size of an object already created, as suggested a
couple messages ago..". At least myself I did not make such
irresponsible statements.

“Please do not confuse with some proposals that had as object dynamically growing objects or anything in that direction.” You said this in reference to N3875, and I just wanted to clarify that N3875 (and various related or similar papers) don’t have that either, yet they have run into trouble.

I don’t think it makes much difference whether a size is determined at link/load time or upon a function call.

Post by mobi phil
For the rest if you bother, just go through the thread.

This thread has become insanely long. Where am I supposed to start? Perhaps now is a good time to start compiling your findings into a comprehensive paper.

[mobi phil]

Post by David Krauss
I don’t think it makes much difference whether a size is determined at link/load time or upon a function call.

The statement is ambiguous. While it will obviously make difference
for the compiler, as the feature has to be implemented, it will also
make huge difference if a function call could be replaced by linker
with a "link time constant" that is link-time-inlined. For some
reasons LLVM guys are working hard on such optimizations. It is easy
to reject by blaming potential run-time performance degradation, but
if that is excluded then
If C++ committee thinks only in terms of technology of 20 years ago,
indeed there will be plenty of reasons to refuse anything.

Paper will be ready soon, please do not blame me for the thread
becoming long, could have ignored all replies but that was obviously
not my interest.