Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: Bricolage: users

force repreview of media and stories

 

 

Bricolage users RSS feed   Index | Next | Previous | View Threaded


terence.bodola at cbsparamount

Aug 4, 2008, 10:24 AM

Post #1 of 15 (866 views)
Permalink
force repreview of media and stories

We wish to wipe our preview server web root and have bric regenerate media
and story previews only when they are repreviewed (space saving).

Right now if we delete any media from the preview web root we just get a 404
on preview in bric...it isn't moved again from
/usr/local/bricolage/comp/data/media subdirs to preview dir on click of
preview url. Is there any way effect this?

Also, is there a scripted way to move all from comp/data/media/... to proper
preview directories...so we could like en masse use bric to move our preview
dir (dupe it without actually copying the existing one which may be
imperfect)? Can we do the same thing with stories...effect a programmatic
repreview of all of them to a certain web root -- this way we can dupe and
then link web root to new mount (not interested in rsync or scp of preview
dir...wanna make sure we get anything in media and all stories that were
previewed at any point in past, as some of older material was probably
removed from preview web root manually to save space...but would like to
recreate now in new mount so no 404 on preview).

Terence


david at kineticode

Aug 4, 2008, 10:38 AM

Post #2 of 15 (844 views)
Permalink
Re: force repreview of media and stories [In reply to]

On Aug 4, 2008, at 10:24, bodola, terence wrote:

> We wish to wipe our preview server web root and have bric regenerate
> media
> and story previews only when they are repreviewed (space saving).

You must not have much space…

> Right now if we delete any media from the preview web root we just
> get a 404
> on preview in bric...it isn't moved again from
> /usr/local/bricolage/comp/data/media subdirs to preview dir on click
> of
> preview url. Is there any way effect this?

AUTO_PREVIEW_MEDIA = 1

> Also, is there a scripted way to move all from comp/data/media/...
> to proper
> preview directories...so we could like en masse use bric to move our
> preview
> dir (dupe it without actually copying the existing one which may be
> imperfect)?

bric_soap media list_ids | bric_soap workflow publish --to-preview -

> Can we do the same thing with stories...effect a programmatic
> repreview of all of them to a certain web root -- this way we can
> dupe and
> then link web root to new mount

Yes, same as above. You just have to pipe in story IDs instead of
media IDs.

> (not interested in rsync or scp of preview
> dir...wanna make sure we get anything in media and all stories that
> were
> previewed at any point in past, as some of older material was probably
> removed from preview web root manually to save space...but would
> like to
> recreate now in new mount so no 404 on preview).

It will be expensive. I don't recommend removing files from a preview
root anymore than I'd recommend removing them from a production root.

Best,

David


terence.bodola at cbsparamount

Aug 13, 2008, 2:28 PM

Post #3 of 15 (819 views)
Permalink
Re: force repreview of media and stories [In reply to]

this seems to have worked to re-preview only for the primary output channel
of my media and stories. how can i re-preview for all output channels on
each story and media (bric 1.10.2)?

also, can i simply blow out /usr/local/bricolage/tmp/* or will this cause
problems? i'm not sure if it is this re-preview of everything that i did,
but tmp is now full of pretty much all data, and bric_clean_tmp seems to
only remove session and lock data.

terence


On 8/4/08 10:38 AM, "David E. Wheeler" <david[at]kineticode.com> wrote:

> On Aug 4, 2008, at 10:24, bodola, terence wrote:
>
>> We wish to wipe our preview server web root and have bric regenerate
>> media
>> and story previews only when they are repreviewed (space saving).
>
> You must not have much spaceŠ
>
>> Right now if we delete any media from the preview web root we just
>> get a 404
>> on preview in bric...it isn't moved again from
>> /usr/local/bricolage/comp/data/media subdirs to preview dir on click
>> of
>> preview url. Is there any way effect this?
>
> AUTO_PREVIEW_MEDIA = 1
>
>> Also, is there a scripted way to move all from comp/data/media/...
>> to proper
>> preview directories...so we could like en masse use bric to move our
>> preview
>> dir (dupe it without actually copying the existing one which may be
>> imperfect)?
>
> bric_soap media list_ids | bric_soap workflow publish --to-preview -
>
>> Can we do the same thing with stories...effect a programmatic
>> repreview of all of them to a certain web root -- this way we can
>> dupe and
>> then link web root to new mount
>
> Yes, same as above. You just have to pipe in story IDs instead of
> media IDs.
>
>> (not interested in rsync or scp of preview
>> dir...wanna make sure we get anything in media and all stories that
>> were
>> previewed at any point in past, as some of older material was probably
>> removed from preview web root manually to save space...but would
>> like to
>> recreate now in new mount so no 404 on preview).
>
> It will be expensive. I don't recommend removing files from a preview
> root anymore than I'd recommend removing them from a production root.
>
> Best,
>
> David
>


david at kineticode

Aug 13, 2008, 7:45 PM

Post #4 of 15 (820 views)
Permalink
Re: force repreview of media and stories [In reply to]

On Aug 13, 2008, at 14:28, bodola, terence wrote:

> this seems to have worked to re-preview only for the primary output
> channel
> of my media and stories. how can i re-preview for all output
> channels on
> each story and media (bric 1.10.2)?

Hrm. Not via SOAP, I guess. :-)

> also, can i simply blow out /usr/local/bricolage/tmp/* or will this
> cause
> problems? i'm not sure if it is this re-preview of everything that
> i did,
> but tmp is now full of pretty much all data, and bric_clean_tmp
> seems to
> only remove session and lock data.

See:

http://www.bricolage.cc/docs/current/api/?Bric::Admin

Then click "Application" under "System Administration" (near the
bottom of the table of contents).

HTH,

David


terence.bodola at cbsparamount

Aug 14, 2008, 6:35 AM

Post #5 of 15 (816 views)
Permalink
RE: force repreview of media and stories [In reply to]

so then my next best option would be lwp or similar for repreviewing all output channels for stories and media?

i see that it says in the api browser that bric_clean_tmp cleans all files in tmp older than 12 hours...but my installation doesn't seem to do this when bric_clean_tmp is croned...and i don't think it is some kind of atime issue...just doesn't seem to clean anything (except maybe lock/session). so you are saying it is find to blow out everything in /usr/local/bricolage/tmp/ ? i realize this will break all user sessions, so should boot everyone from system first or not blow out lock/session files...but otherwise ok?

also, got a forbidden for the link at the bottom of that System Administration -> Application section...the link to: http://www.bricolage.cc/docs/current/api/Bric/?bric_clean_tmp

Thanks,
Terence Bodola
terence.bodola[at]cbsparamount.com
323-956-4813



-----Original Message-----
From: David E. Wheeler [mailto:david[at]kineticode.com]
Sent: Wed 8/13/2008 7:45 PM
To: users[at]lists.bricolage.cc
Subject: Re: force repreview of media and stories

On Aug 13, 2008, at 14:28, bodola, terence wrote:

> this seems to have worked to re-preview only for the primary output
> channel
> of my media and stories. how can i re-preview for all output
> channels on
> each story and media (bric 1.10.2)?

Hrm. Not via SOAP, I guess. :-)

> also, can i simply blow out /usr/local/bricolage/tmp/* or will this
> cause
> problems? i'm not sure if it is this re-preview of everything that
> i did,
> but tmp is now full of pretty much all data, and bric_clean_tmp
> seems to
> only remove session and lock data.

See:

http://www.bricolage.cc/docs/current/api/?Bric::Admin

Then click "Application" under "System Administration" (near the
bottom of the table of contents).

HTH,

David


david at kineticode

Aug 14, 2008, 8:34 AM

Post #6 of 15 (815 views)
Permalink
Re: force repreview of media and stories [In reply to]

On Aug 14, 2008, at 06:35, bodola, terence wrote:

> so then my next best option would be lwp or similar for repreviewing
> all output channels for stories and media?

Yeah. I find that it's pretty rare that media are in multiple OCs,
though, since no templates get applied to them. Why are yours in
multiple OCs?

> i see that it says in the api browser that bric_clean_tmp cleans all
> files in tmp older than 12 hours...but my installation doesn't seem
> to do this when bric_clean_tmp is croned...and i don't think it is
> some kind of atime issue...just doesn't seem to clean anything
> (except maybe lock/session). so you are saying it is find to blow
> out everything in /usr/local/bricolage/tmp/ ?

No, I didn't say that. What's in that directory?

It probably is fine, though all your users will be logged out
afterward, and you should restart Bricolage after you delete it, too.
Even better, shut it down, then clear out everything, then start up
again.

> i realize this will break all user sessions, so should boot everyone
> from system first or not blow out lock/session files...but otherwise
> ok?

Probably. But why do you want to do this?

> also, got a forbidden for the link at the bottom of that System
> Administration -> Application section...the link to: http://www.bricolage.cc/docs/current/api/Bric/?bric_clean_tmp

Yeah, known issue. I'll be revamping the doc browser in the next few
weeks and fixing those bugs.

Best,

David


terence.bodola at cbsparamount

Aug 14, 2008, 9:10 AM

Post #7 of 15 (811 views)
Permalink
Re: force repreview of media and stories [In reply to]

you're right. it's just our stories that have multiple ocs. what is
happening is that since the non-primary story ocs haven't been repreviewed
(and having blown out our preview web server dir those original preview
files are gone) we are having preview problems with other stories that use
those story oc preview files.

can i repreview these from the other direction...like, can i repreview all
stories in an oc (previewing in that oc not the primary) using bric_soap?

so in tmp there is:
/usr/local/bricolage/tmp/usr/local/bricolage/comp/data/media <--that has all
the media dirs under it
/usr/local/bricolage/tmp/usr/local/bricolage/data/burn <--that has preview
and stage dirs under it, and they have the oc_* dirs under them

i want to blow out tmp b/c it is large...if the files will just be recreated
on use and nothing bad comes of it. i am a little wary now b/c of the pain
blowing out preview web dir caused. i thought media would just rebuild in
preview web dir upon repreview of stories that use the media, but that's not
right, and so when previewing a story you would now first need to go
repreview all media and stories used/included in it individually first (and
possibly non-primary ocs for those stories as well). you helped me fix this
problem for media with a bric_soap scripted repreview (thank you for that),
and it is mostly fixed for stories (it is fixed for primary oc)...just not
completely.

right now i backup the bric tmp dir (b/c i don't know if i can not do it)
and so it takes up a lot of space not just on the bric server but on the
backup servers as well (and since it changes a good bit i think it is making
our daily incremental backups much larger than they need be).

terence


On 8/14/08 8:34 AM, "David E. Wheeler" <david[at]kineticode.com> wrote:

> On Aug 14, 2008, at 06:35, bodola, terence wrote:
>
>> so then my next best option would be lwp or similar for repreviewing
>> all output channels for stories and media?
>
> Yeah. I find that it's pretty rare that media are in multiple OCs,
> though, since no templates get applied to them. Why are yours in
> multiple OCs?
>
>> i see that it says in the api browser that bric_clean_tmp cleans all
>> files in tmp older than 12 hours...but my installation doesn't seem
>> to do this when bric_clean_tmp is croned...and i don't think it is
>> some kind of atime issue...just doesn't seem to clean anything
>> (except maybe lock/session). so you are saying it is find to blow
>> out everything in /usr/local/bricolage/tmp/ ?
>
> No, I didn't say that. What's in that directory?
>
> It probably is fine, though all your users will be logged out
> afterward, and you should restart Bricolage after you delete it, too.
> Even better, shut it down, then clear out everything, then start up
> again.
>
>> i realize this will break all user sessions, so should boot everyone
>> from system first or not blow out lock/session files...but otherwise
>> ok?
>
> Probably. But why do you want to do this?
>
>> also, got a forbidden for the link at the bottom of that System
>> Administration -> Application section...the link to:
>> http://www.bricolage.cc/docs/current/api/Bric/?bric_clean_tmp
>
> Yeah, known issue. I'll be revamping the doc browser in the next few
> weeks and fixing those bugs.
>
> Best,
>
> David
>


simonw at digitalcraftsmen

Aug 14, 2008, 9:40 AM

Post #8 of 15 (807 views)
Permalink
Re: force repreview of media and stories [In reply to]

bodola, terence wrote:
> you're right. it's just our stories that have multiple ocs. what is
> happening is that since the non-primary story ocs haven't been repreviewed
> (and having blown out our preview web server dir those original preview
> files are gone) we are having preview problems with other stories that use
> those story oc preview files.

We have a similar problem which I mentioned a while back. Unfortunately
we haven't had time to fix it yet.

I'm still unclear why previewing a story only previews in the primary OC
and not in all of them. Why would you not want that ?

S.

--
Digital Craftsmen Ltd
Exmouth House, 3 Pine Street, London. EC1R 0JH
t 020 7183 1410 f 020 7099 5140 m 07951 758698
w http://www.digitalcraftsmen.net/


david at kineticode

Aug 14, 2008, 10:46 AM

Post #9 of 15 (808 views)
Permalink
Re: force repreview of media and stories [In reply to]

On Aug 14, 2008, at 09:40, Simon Wilcox wrote:

> We have a similar problem which I mentioned a while back.
> Unfortunately we haven't had time to fix it yet.
>
> I'm still unclear why previewing a story only previews in the
> primary OC and not in all of them. Why would you not want that ?

Hysterical reasons. Fine with me to change it, at least when doing
bulk previews.

Best,

David


terence.bodola at cbsparamount

Aug 14, 2008, 10:49 AM

Post #10 of 15 (815 views)
Permalink
Re: force repreview of media and stories [In reply to]

point me in a direction and i'll give the code change a shot.


On 8/14/08 10:46 AM, "David E. Wheeler" <david[at]kineticode.com> wrote:

> On Aug 14, 2008, at 09:40, Simon Wilcox wrote:
>
>> We have a similar problem which I mentioned a while back.
>> Unfortunately we haven't had time to fix it yet.
>>
>> I'm still unclear why previewing a story only previews in the
>> primary OC and not in all of them. Why would you not want that ?
>
> Hysterical reasons. Fine with me to change it, at least when doing
> bulk previews.
>
> Best,
>
> David
>


david at kineticode

Aug 18, 2008, 10:16 AM

Post #11 of 15 (796 views)
Permalink
Re: force repreview of media and stories [In reply to]

On Aug 14, 2008, at 10:49, bodola, terence wrote:

> point me in a direction and i'll give the code change a shot.

I think it's this block in Bric::SOAP::Workflow:

if ($preview) {
# Just preview it.
push @published, name( "$type\_id" => $id )
if $burner->preview($obj, $type, get_user_id);

You should change it to iterate over all of the output channels, like
so:

if ($preview) {
# Just preview it.
$burner->preview($obj, $type, get_user_id, $_->get_id)
for $obj->get_output_channels;
push @published, name( "$type\_id" => $id );

Try that.

Best,

David


terence.bodola at cbsparamount

Aug 18, 2008, 11:26 AM

Post #12 of 15 (798 views)
Permalink
Re: force repreview of media and stories [In reply to]

awesome...but shouldn't there still be some sort of if clause around
$burner->preview before pushing into published array...like there was
originally?


On 8/18/08 10:16 AM, "David E. Wheeler" <david[at]kineticode.com> wrote:

> On Aug 14, 2008, at 10:49, bodola, terence wrote:
>
>> point me in a direction and i'll give the code change a shot.
>
> I think it's this block in Bric::SOAP::Workflow:
>
> if ($preview) {
> # Just preview it.
> push @published, name( "$type\_id" => $id )
> if $burner->preview($obj, $type, get_user_id);
>
> You should change it to iterate over all of the output channels, like
> so:
>
> if ($preview) {
> # Just preview it.
> $burner->preview($obj, $type, get_user_id, $_->get_id)
> for $obj->get_output_channels;
> push @published, name( "$type\_id" => $id );
>
> Try that.
>
> Best,
>
> David
>


david at kineticode

Aug 18, 2008, 11:55 AM

Post #13 of 15 (792 views)
Permalink
Re: force repreview of media and stories [In reply to]

On Aug 18, 2008, at 11:26, bodola, terence wrote:

> awesome...but shouldn't there still be some sort of if clause around
> $burner->preview before pushing into published array...like there was
> originally?

No. If the preview fails, it'll throw an exception.

/me looks…

Oh, no, I'm lying. It will return false if there are no files to
preview. So, um, try this:

if ($preview) {
# Just preview it.
push @published, name( "$type\_id" => $id ) if grep {
$burner->preview($obj, $type, get_user_id, $_->get_id)
} $obj->get_output_channels

Best,

David


terence.bodola at cbsparamount

Aug 19, 2008, 8:44 AM

Post #14 of 15 (783 views)
Permalink
Re: force repreview of media and stories [In reply to]

worked like a charm. thanks man.


On 8/18/08 11:55 AM, "David E. Wheeler" <david[at]kineticode.com> wrote:

> On Aug 18, 2008, at 11:26, bodola, terence wrote:
>
>> awesome...but shouldn't there still be some sort of if clause around
>> $burner->preview before pushing into published array...like there was
>> originally?
>
> No. If the preview fails, it'll throw an exception.
>
> /me looksŠ
>
> Oh, no, I'm lying. It will return false if there are no files to
> preview. So, um, try this:
>
> if ($preview) {
> # Just preview it.
> push @published, name( "$type\_id" => $id ) if grep {
> $burner->preview($obj, $type, get_user_id, $_->get_id)
> } $obj->get_output_channels
>
> Best,
>
> David


Terence Bodola
terence.bodola[at]cbsparamount.com
o: 323-956-4813
c: 520-429-4488


david at kineticode

Aug 19, 2008, 10:19 AM

Post #15 of 15 (778 views)
Permalink
Re: force repreview of media and stories [In reply to]

On Aug 19, 2008, at 08:44, bodola, terence wrote:

> worked like a charm. thanks man.

Committed to trunk in r8269.

Best,

David

Bricolage users RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact lists@gossamer-threads.com
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.