BTS

Issue1284

Title check4progs() add option to operta in silent mode
Priority wish Status resolved
Superseder Nosy List Xk2c, ft
Assigned To ft Topics

Created on 2013-11-08.15:18:27 by Xk2c, last changed 2013-12-24.12:30:55 by mika.

Files
File name Uploaded Type Edit Remove
0001-script-functions-Set-filetype-to-sh-with-vim.patch ft, 2013-11-17.14:44:01 text/x-diff
0002-check4progs-Change-testing-from-which-to-iterating-o.patch ft, 2013-11-17.14:44:01 text/x-diff
check4progs.PATHloop Xk2c, 2013-11-17.13:33:23 application/octet-stream
check4progs.PATHloop Xk2c, 2013-11-17.14:13:49 text/plain
script-functions Xk2c, 2013-11-10.17:04:07 application/octet-stream
script-functions Xk2c, 2013-11-12.16:30:56 application/octet-stream
script-functions Xk2c, 2013-11-17.14:13:49 text/plain
script-functions.patch Xk2c, 2013-11-17.14:13:49 text/x-diff
Messages
msg4710 (view) Author: mika Date: 2013-12-24.12:30:55
We believe that your issue has been closed by the upload of
Version 0.9.2 of grml-etc-core from Michael Prokop <mika@grml.org>.
The explanation is attached below

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Format: 1.8
Date: Tue, 24 Dec 2013 13:12:42 +0100
Source: grml-etc-core
Binary: grml-etc-core
Architecture: source all
Version: 0.9.2
Distribution: grml-testing
Urgency: low
Maintainer: Michael Prokop <mika@grml.org>
Changed-By: Michael Prokop <mika@grml.org>
Description: 
 grml-etc-core - core etcetera files for the grml system
Changes: 
 grml-etc-core (0.9.2) grml-testing; urgency=low
 .
   [ Michael Prokop ]
   * [4bbd12c] script-functions: correct tabstop setting in vim modeline
   * [a74236f] Bump Standards-Version to 3.9.5
 .
   [ Thilo Six ]
   * [6b83c3f] script-functions: Set filetype to "sh" with vim
   * [5f9a26a] check4progs: Change testing from ‘which’ to iterating
     over $PATH [Closes: issue1284]
 .
   [ Frank Terbeck ]
   * [eb08ffe] Update hl and _hl_complete to work with current
     ‘highlight’ versions
Checksums-Sha1: 
 2a2e2a299a4986db840b2f8dde6ccf0b00c030d5 945 grml-etc-core_0.9.2.dsc
 883bdfa5c04649058b642990c314aef2bfeda3f9 151585 grml-etc-core_0.9.2.tar.gz
 32453ff63a25433c1a6d2bddf5c3454cea9dee35 139116 grml-etc-core_0.9.2_all.deb
Checksums-Sha256: 
 ece2021bf07b1f47515a657e6b939ce26aa07b6bc861e1ce07cbe577ed2d9ae2 945 grml-etc-core_0.9.2.dsc
 e7815fe10d0a70c5452438f1db28e3144e5f2a33475532acc999d8da7075d7e9 151585 grml-etc-core_0.9.2.tar.gz
 c9d8a0370b2da6967948e97f9e057b590882407fe5533cf075db829f42bf6694 139116 grml-etc-core_0.9.2_all.deb
Files: 
 946e7566f0051f88b0d8f1ae4cd892c8 945 grml optional grml-etc-core_0.9.2.dsc
 0720afdbcb0e1c59a309aae097e16d68 151585 grml optional grml-etc-core_0.9.2.tar.gz
 eb96a8f4188973c595ac3ffbf31215b6 139116 grml optional grml-etc-core_0.9.2_all.deb

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.15 (GNU/Linux)

iEYEARECAAYFAlK5eqAACgkQ2N9T+zficuhvgQCePtvX8P3lhbJ6AafMuzvcvgQm
6G0An33YxB7oz2jitjt5IButdPGMZu60
=lsRb
-----END PGP SIGNATURE-----
msg4688 (view) Author: ft Date: 2013-11-17.14:44:01
Thilo Six wrote:
[...]
> Thats it! It has been tested in bash (4.2.45) and zsh (5.0.2) and verified that
> it works as aspected.

Great!

[...]
> In script-functions there is one more change the discussed allready.
> I changed the vim modeline:
>
> -# vim:foldmethod=marker tw=80 ai expandtab shiftwidth=2 tabstop=8
> +# vim:foldmethod=marker tw=80 ai expandtab shiftwidth=2 tabstop=8 ft=sh
[...]

Since this is another unrelated change, I'd usually request two separate
patches. But since you're getting to learn git (and probably version
control as a whole) just now, I'll split them up manually. 

Actually, we could split this up even more, since the does two things:

    - Add the --silent etc. options
    - Switch from ‘which’ to looping over $PATH.

But then again I'm lazy, so lets not over-engineer this. ;)

For the record, I'll attach patches for the two commits, that I will
push in just a minute. If the patches are formatted like this (the git
format-patch utility helps with this) the integration work on our side
is minimal. ;)

Regards, Frank
msg4687 (view) Author: Xk2c Date: 2013-11-17.14:13:49
Hello Frank,

Excerpt from Frank Terbeck:

-- <snip> --
> Err, heh. Well, looks like a typo snuck in. That should obviously be
> VERSION, not VERISON. So:
> 
>     [ -n "$ZSH_VERSION" ] && emulate -L sh

Thats it! It has been tested in bash (4.2.45) and zsh (5.0.2) and verified that
it works as aspected.

Attached are: the standalone check4progs(), modified script-functions and the
patch against current git verion.


In script-functions there is one more change the discussed allready.
I changed the vim modeline:

-# vim:foldmethod=marker tw=80 ai expandtab shiftwidth=2 tabstop=8
+# vim:foldmethod=marker tw=80 ai expandtab shiftwidth=2 tabstop=8 ft=sh

This way script-functions gets correct highlighting and stuff in vim out of the
box. (filetype is conf otherwise).


It has been a pleasure to work with you Frank!


kind regards,

     Thilo


-- 
$ \grep -Thilo '[a-zA-Z0-9._-]\+()' ~/.bashrc
msg4686 (view) Author: ft Date: 2013-11-17.13:59:42
Thilo Six wrote:
[...]
>>     [ -n "$ZSH_VERISON" ] && emulate -L sh
>
> I tried this, same effect as before - it does not work.
> I guess the reason is the same, since "emulate -L sh" isn't activated
> '$ZSH_VERISON' will not be evaluated correctly.
> Setting "emulate -L sh" prior to running the function in zsh fixes this.

Err, heh. Well, looks like a typo snuck in. That should obviously be
VERSION, not VERISON. So:

    [ -n "$ZSH_VERSION" ] && emulate -L sh
msg4685 (view) Author: Xk2c Date: 2013-11-17.13:50:51
Excerpt from Frank Terbeck:-- <snip> --
>> BUT the funny thing with this is it does work in bash but not in zsh. :/
>>
>> e.g.:
>> ~ % which cp
>> /bin/cp
>> ~ % check4progs foo cp
>> foo: binary not found
>> cp: binary not found
>>
>> This is with unmodfied debian zsh. I guess some "setopt" is missing here?
> 
> 
> Ah yes. Good catch.
> 
> The reason for this is, that zsh in its native mode of operation is very
> much *not* a POSIX shell (in particular, $foo is by default expanded as
> if you wrote "$foo" in a POSIX shell). Add the following line right at
> the top of the function. It'll enable zsh's POSIX-emulation mode locally
> in the function:
> 
>     [ -n "$ZSH_VERISON" ] && emulate -L sh

I tried this, same effect as before - it does not work.
I guess the reason is the same, since "emulate -L sh" isn't activated
'$ZSH_VERISON' will not be evaluated correctly.
Setting "emulate -L sh" prior to running the function in zsh fixes this.

> That should do the trick. The code should otherwise run in everything,
> that has ‘local’ functionality, which is basically every other POSIXly
> looking shell (like bash, ksh93, (d)ash, mksh, pksh, posh etc).
> 
> Regards, Frank
> 
> _____________________________________
> GRML issue tracker <bts@bts.grml.org>
> <http://bts.grml.org/grml/issue1284>
> _____________________________________
> 


-- 
$ \grep -Thilo '[a-zA-Z0-9._-]\+()' ~/.bashrc
msg4684 (view) Author: ft Date: 2013-11-17.13:41:46
Thilo Six wrote:
> That is nice! I added back the logic to suppress output at user will.
>
> BUT the funny thing with this is it does work in bash but not in zsh. :/
>
> e.g.:
> ~ % which cp
> /bin/cp
> ~ % check4progs foo cp
> foo: binary not found
> cp: binary not found
>
> This is with unmodfied debian zsh. I guess some "setopt" is missing here?


Ah yes. Good catch.

The reason for this is, that zsh in its native mode of operation is very
much *not* a POSIX shell (in particular, $foo is by default expanded as
if you wrote "$foo" in a POSIX shell). Add the following line right at
the top of the function. It'll enable zsh's POSIX-emulation mode locally
in the function:

    [ -n "$ZSH_VERISON" ] && emulate -L sh

That should do the trick. The code should otherwise run in everything,
that has ‘local’ functionality, which is basically every other POSIXly
looking shell (like bash, ksh93, (d)ash, mksh, pksh, posh etc).

Regards, Frank
msg4683 (view) Author: Xk2c Date: 2013-11-17.13:33:23
Hello Frank,

That is nice! I added back the logic to suppress output at user will.

BUT the funny thing with this is it does work in bash but not in zsh. :/

e.g.:
~ % which cp
/bin/cp
~ % check4progs foo cp
foo: binary not found
cp: binary not found

This is with unmodfied debian zsh. I guess some "setopt" is missing here?

I attach the tested "check4progs.PATHloop" for ease of further testing.
msg4682 (view) Author: ft Date: 2013-11-17.11:13:03
Hey,

Thilo Six wrote:
> i am again. Hope you still like reading mail from me.  ;)
> Frank suggested using shell builtin instead of '\which'.
>
> To my experience this noticeable speeds thinks up, which  ;)
> is logically since changing '\which foo' to 'builtin which foo' 
> reduces filesystemseeks by 50%.
> Since 'builtin which foo' is zsh specific this resulted in quite a big patch
> linewise.
> So if you are interested here is the updated patch for check4progs().

Yeah, also: fork()ing for external programs is much more expensive then
calling built-in programs.

I think I once wrote a small utility for one of our scripts, that just
checked $PATH and didn't call which, type or whatever at all. I can't
remember which script that was though.

So here's another version:

#+BEGIN_SRC shell-script
check4progs() {
    local RTN=0
    local ARG
    local oldifs="$IFS"
    local d found

    while [ $# -gt 0 ]; do
        ARG="$1"
        shift

        found=0
        IFS=:
        for d in $PATH; do
            if [ -x "$d/$ARG" ]; then
                found=1
                break
            fi
        done
        IFS="$oldifs"

        # check for availability
        if [ "$found" -eq 0 ]; then
            printf "%s: binary not found\n" "$ARG" >&2
            RTN=1
        fi
    done

    # return non zero, if at least one prog is missing!
    return $RTN
}
#+END_SRC shell-script

That should be fairly portable (the ‘local’ instructions are not
strictly POSIX, but at least debian demands its /bin/sh to support it)
and avoids special-casing for different shells.

Feel free to benchmark this against your versions. Calling a built-in
_might_ be a bit quicker then looping over $PATH entries, but I don't
think it will make much of a difference, unless $PATH has a _lot_ of
entries.

In any case, I think generality would trump the last bits of performance
(I'm pretty positive iterating over $PATH would still beat calling an
external program). Because, if performance is the most important
measurement for something, implementing a solution as a shell-script is
wrong anyway. :-)

Note that I just typed this into my mail-client. I didn't even test if
it works, nor did I perform any bench-marking. But I think it'll give
you an idea, of what a general solution could look like.

Regards, Frank
msg4679 (view) Author: Xk2c Date: 2013-11-12.16:30:56
Hello

i am again. Hope you still like reading mail from me.  ;)
Frank suggested using shell builtin instead of '\which'.

To my experience this noticeable speeds thinks up, which  ;)
is logically since changing '\which foo' to 'builtin which foo' 
reduces filesystemseeks by 50%.
Since 'builtin which foo' is zsh specific this resulted in quite a big patch
linewise.
So if you are interested here is the updated patch for check4progs().

Regards,
Thilo
msg4678 (view) Author: mika Date: 2013-11-12.10:09:41
* Frank Terbeck wrote in grml's BTS on 20131110 / 17:42:
> Thilo Six wrote:
> > PS. now i figured why my patches are so bad intended.

> > This vim modeline:
> > # vim:foldmethod=marker tw=80 ai expandtab shiftwidth=2 tabstop=2

> > should really be:
> > # vim:foldmethod=marker tw=80 ai expandtab shiftwidth=2 softtabstop=2

> > The value of &tabstop should allways be 8.

> Fun. Since I switched my main editor to emacs I don't get thrown off by
> that anymore - obviously. :)

> ‘tabstop = 2’ does sound wrong indeed. Mika?

Yes, no idea why I chose this bad default, sorry.
I've just fixed that
[https://github.com/grml/grml-etc-core/commit/4bbd12cabf9c09c9070b1d5f3eca4b11b70b7bba].

Thanks for raising that!

regards,
-mika-
msg4677 (view) Author: Xk2c Date: 2013-11-10.17:04:07
> There are still two tabs in the newly introduced comment.

i fixed that.

> Sorry for being a bit unresponsive, got a lot on my plate these days.

I think you are the opposite of unresponsive. Thanks!
msg4676 (view) Author: ft Date: 2013-11-10.16:42:52
Thilo Six wrote:
> PS. now i figured why my patches are so bad intended.
>
> This vim modeline:
> # vim:foldmethod=marker tw=80 ai expandtab shiftwidth=2 tabstop=2
>
> should really be:
> # vim:foldmethod=marker tw=80 ai expandtab shiftwidth=2 softtabstop=2
>
> The value of &tabstop should allways be 8.

Fun. Since I switched my main editor to emacs I don't get thrown off by
that anymore - obviously. :)

‘tabstop = 2’ does sound wrong indeed. Mika?
msg4675 (view) Author: ft Date: 2013-11-10.16:39:51
Thilo Six wrote:
> Thilo Six <tech@xk2c.de> added the comment:
>
> I have updated the attached files.

There are still two tabs in the newly introduced comment.
msg4674 (view) Author: ft Date: 2013-11-10.16:39:49
Sorry for being a bit unresponsive, got a lot on my plate these days.

Thilo Six wrote:
[...]
> sometime i have the need to check for avail progs but i do not want
> any output. I just want the return code. Attached patch adds this to
> check4progs().

Alright. I don't see were the variables with the colour escape sequences
are coming from. I'd lean towards not having coloured messages in these
utilities.

> PS. Regarding to:
>> The only thing you could have done better, was to send a patch with commit
>> message generated by "git format-patch". That way I could have applied this
>> using one command.
>
> I have no experienced what to do here. So i sent this patch "as is".

Well, we keep our code in separate git repositories. If you don't know
git yet, there's a free book only at: http://git-scm.com/book

We also keep some information about contributing to grml at:
http://grml.github.io/

The goal would be to create one email message per version control
commit. You can send those to grml-etc-core@grml.org or just attach them
to the issue you opened. Then git has some utilities to import them
again at my end with virtually no manual work at all.

I wrote about that work-flow a while back:
http://bewatermyfriend.org/posts/2007/10-02.17-22-07-computer.html

Regards, Frank

-- 
In protocol design, perfection has been reached not when there is
nothing left to add, but when there is nothing left to take away.
                                                  -- RFC 1925
msg4673 (view) Author: Xk2c Date: 2013-11-09.22:36:50
PS. now i figured why my patches are so bad intended.

This vim modeline:
# vim:foldmethod=marker tw=80 ai expandtab shiftwidth=2 tabstop=2

should really be:
# vim:foldmethod=marker tw=80 ai expandtab shiftwidth=2 softtabstop=2

The value of &tabstop should allways be 8.
msg4672 (view) Author: Xk2c Date: 2013-11-09.21:57:24
I have updated the attached files.
msg4671 (view) Author: Xk2c Date: 2013-11-08.15:32:08
When i look a this patch the indenting is broken.
I also attach the "standanlone" version where indenting is right.
msg4670 (view) Author: Xk2c Date: 2013-11-08.15:18:27
Hello Frank
Hello graml Team,

this is somewhat a follow up of
http://bts.grml.org/grml/issue1279


sometime i have the need to check for avail progs but i do not want any output.
I just want the return code.
Attached patch adds this to check4progs().

Thx

Thilo

PS. Regarding to:
> The only thing you could have done better, was to send a patch with commit
> message generated by "git format-patch". That way I could have applied this
> using one command.

I have no experienced what to do here. So i sent this patch "as is".

> Yes, I'm lazy. ;)

No Problem. /me 2.  ;P
History
Date User Action Args
2013-12-24 12:30:55mikasetstatus: fixed-in-git -> resolved
messages: + msg4710
2013-11-17 14:51:03ftsetstatus: chatting -> fixed-in-git
2013-11-17 14:44:01ftsetfiles: + 0001-script-functions-Set-filetype-to-sh-with-vim.patch, 0002-check4progs-Change-testing-from-which-to-iterating-o.patch
messages: + msg4688
2013-11-17 14:13:49Xk2csetfiles: + check4progs.PATHloop, script-functions, script-functions.patch
messages: + msg4687
2013-11-17 13:59:42ftsetmessages: + msg4686
2013-11-17 13:50:51Xk2csetmessages: + msg4685
2013-11-17 13:41:46ftsetmessages: + msg4684
2013-11-17 13:33:23Xk2csetfiles: + check4progs.PATHloop
messages: + msg4683
2013-11-17 11:13:03ftsetmessages: + msg4682
2013-11-12 16:30:56Xk2csetfiles: + script-functions
messages: + msg4679
2013-11-12 10:09:41mikasetmessages: + msg4678
2013-11-10 17:04:07Xk2csetfiles: + script-functions
messages: + msg4677
2013-11-10 17:01:44Xk2csetfiles: - script-functions
2013-11-10 16:42:52ftsetmessages: + msg4676
2013-11-10 16:39:51ftsetmessages: + msg4675
2013-11-10 16:39:50ftsetmessages: + msg4674
2013-11-09 22:36:50Xk2csetmessages: + msg4673
2013-11-09 22:17:14Xk2csetfiles: + script-functions
2013-11-09 22:11:25Xk2csetfiles: - script-functions
2013-11-09 21:57:24Xk2csetfiles: + script-functions
messages: + msg4672
2013-11-09 21:48:43Xk2csetfiles: - script-functions
2013-11-09 21:48:40Xk2csetfiles: - check4prog
2013-11-08 15:32:08Xk2csetfiles: + check4prog
status: unread -> chatting
messages: + msg4671
2013-11-08 15:18:27Xk2ccreate